0.50b 1pass VBR problem

[Momotte]

Hi!

Just tried 1-pass VBR on LOTR ROTK and the resulting size was like 6gb! In the analysis pass it worked properly as it tried with q=47, q=49 but when it was time to build the .ecl, it used q=17 in it(which is too much quality).

[Joergen]

Then again I tried 3pass (with progressive encoding) for that one and the quality wasnt satisfactory to me, so I had to squeeze the menu abit with shrink and use the 320MB 2ch DD track instead of the 630MB DD 5.1. Of course 2.500kbit average is really pushing it.

Extras disc came out pefectly without tweaking, which wouldhave taken days to achieve by conventional methods.

btw. remapping the #2 audio to play as #1 by default can be done via the latest dvdshrink preferences->logical remapping of audio streams. I learnt this only today, dvdshrink never ceases to amaze me. :o

edit: I'd like to add that the R2 theatrical version's encoding (while progressive frames, still encoded interlaced) could be better. There are visible blocks starting with the blue new line cinema animation. But this applies to all previous theatrical versions, especially the first movie was too soft.

Odd that a director from new zealand (PAL land) doesnt force the studio to come out with a razor sharp, spotless release.

edit1195: OMG the more I scroll through the original in dvd2avi the more blocks I see in many scenes. They're subtle, sure, and not visible in motion unless you have the latest 46" plasma from pioneer hooked to component video, but still there. I can expect this from matrix revolutions but from lotr.. tisk tisk! ;)

[wmansir]

Momotte, so are you saying the .ecl doesn't use what RB says it determined to be the Q value, or did DVD-RB start out testing Q=49 and then eventually settled on Q=19?

OT:
Like everyone else, I'm doing this one now too. I used RB-Opt to reduce the credit's bitrate slightly, add Undot.Deen, set DC prec to 8 and am using the low bitrate matrix. At first I though it was a bit blurry, but comparing a few encoded scenes with the original I see that the original isn't too hot either.

[jdobbs]

Originally posted by Momotte
Hi!

Just tried 1-pass VBR on LOTR ROTK and the resulting size was like 6gb! In the analysis pass it worked properly as it tried with q=47, q=49 but when it was time to build the .ecl, it used q=17 in it(which is too much quality). NTSC or PAL? I did OPV using NTSC on this one and it computed Q=50 as I recall. I'm at a loss as to where 17 could have come from??? DVD-RB uses the final value shown in the Status window... what did it say there?

[jdobbs]

Originally posted by wmansir
Momotte, so are you saying the .ecl doesn't use what RB says it determined to be the Q value, or did DVD-RB start out testing Q=49 and then eventually settled on Q=19?

OT:
Like everyone else, I'm doing this one now too. I used RB-Opt to reduce the credit's bitrate slightly, add Undot.Deen, set DC prec to 8 and am using the low bitrate matrix. At first I though it was a bit blurry, but comparing a few encoded scenes with the original I see that the original isn't too hot either. What kind of action does undot.deen take on the video source? Temporal? Spacial?

[wmansir]

Originally posted by jdobbs
What kind of action does undot.deen take on the video source? Temporal? Spacial?

Undot is spatial. It works by keeping each pixel within the threshold of the eight pixels surrounding it. Pretty simple, and equally fast.

Deen (by default) is both spatial and temporal. It is much like Convolution3D, but optimized for better speed. It averages the pixel to the pixels around it spatialy and temporally, if they are within a given threshold.

Check out this thread for an english translation of the readme, and a basic description of the modes (on page 2):

http://forum.doom9.org/showthread.php?s=&threadid=69052

Also, the default Deen()is Deen("c3d", 0, 10, 12, 3).

Anyway, the overall effect is a slight blur in some places. Mostly visible in uniform colored areas, like skin tone. Areas of high, or moderate contrast are not affected.

[Trahald]

ddogg's love of that combination caused it to grow on me. it slows the encode a bit but results are very nice in low bitrate situations.

[wmansir]

I also like FluxSmooth. It's very smiliar to Undot.Deen and is my standard light filter. From the readme:

One of the fundamental properties of noise is that it's random. One of the fundamental properties of motion is that it's not. This is the premise behind FluxSmooth, which examines each pixel and compares it to the corresponding pixel in the previous and last frame. Smoothing occurs if both the previous frame's value and the next frame's value are greater, or if both are less, than the value in the current frame.

I like to call this a "fluctuating" pixel, then I like to wipe that pixel from existence by averaging it with its neighbours. This is (by default) done in a spatio-temporal manner, in that for each fluctuating pixel its 8 immediate spatial neighbours as well as its 2 temporal neighbours (the abovementioned corresponding pixel from the previous and next frames) are considered for inclusion in the average. If the value of each pixel is within the specified threshold, it is included. If not, it isn't.

It's different from Undot.Deen mostly in that only 'flux' pixels are affected. It's also very simple to use, with only two arguments, Temporal threshold and Spatial threshold.

EDIT to say: If you check it out, don't worry about the problem with CCE mentioned in the readme, I have never encountered it in many many encodes.

[wmansir]

Also, for RotK, I suggest using DVD Shrink on the menu. I did it with maximum compression (and stripped the 2.0 audio from the feature) and got the menu down from 210MB to 77MB, and it looks excellent. DVD-Shrink can really pixilate some menues, but others come out looking great. It's probably a little blurry compared to the original, but without them beside each other, I didn't notice.

[Joergen]

On the R2 nordic version the menu couldnt handle even 80% properly so its quite "optimized" eventhough it's 150MB with only english.

btw if you use the dynamic stream reallocation of dvdshrink, DO NOT remove subtitles (above the one you want to see) or they will be moved up aswell and the language menu stops working.

[Momotte]

jdobbs: It tried q=47 (giving certain amount of sector) then q=49(with different num. sectors) and concluded to use q=17 as the q factor. This is what appeared in the log. Is it normal? Anyways, was way oversized... it is the LOTR3 NTSC...

Thanks,

[jdobbs]

Originally posted by Momotte
jdobbs: It tried q=47 (giving certain amount of sector) then q=49(with different num. sectors) and concluded to use q=17 as the q factor. This is what appeared in the log. Is it normal? Anyways, was way oversized... it is the LOTR3 NTSC...

Thanks, No it's not. I'll review the calculation. Do you still have the status output?

[Momotte]

No I don't but I can retry it when I get back home this afternoon and copy/paste the log file and send it here.

I will also give a try to 0.51 at the same time and see if I can reproduce

[jdobbs]

I'm doing that one right now (with 0.51) to test this out. Did you have any special settings (like half/half) set? Also which version of CCE were you using?

[Momotte]

no special settings except idct=7... there is no need for half-d1 for extras since the first disc only contains the movie(almost). Anyways, I never used half-d1 up to now... I unchecked the french ac3 track(only left the english)...

[jdobbs]

That's interesting. The one I have doesn't have a French track, only two English ones.

[quantum]

@jdobbs
It seems the one-pass feature is number one on your to-do list. Any idea where multi-angle is? How about captions or the sub-title issue?

It seems to me the one-pass feature is interesting, but we can already finish an encode the traditional way albeit with more processing time, so it doesn't add anything we couldn't do before, or fix anything that was broken.

Of course it's up to you and I know some things are more "fun" to work on than others.

[jdobbs]

Originally posted by quantum
@jdobbs
It seems the one-pass feature is number one on your to-do list. Any idea where multi-angle is? How about captions or the sub-title issue?

It seems to me the one-pass feature is interesting, but we can already finish an encode the traditional way albeit with more processing time, so it doesn't add anything we couldn't do before, or fix anything that was broken.

Of course it's up to you and I know some things are more "fun" to work on than others. I guess it could look that way. But it isn't. The amount of time I've spent on OPV is insignificant compared to the last several weeks that I've spent going through the same code over-and-over looking for something that didn't stand out the last 20 times I looked at it related to the non-playing subtitles.

The fact is I can't repeat it and since I really don't change anything on the subtitles I'm not sure where to go next. I obviously have a problem that gets reflected on certain players... but finding it when it never does it for me is almost impossible -- and I don't want to go out and buy a Phillips player just so I can see an error.

[Joergen]

Slightly OT: Well the closed captions problem(?) is not a problem for pal users since the CC feature is not used (I've never seen it mentioned on a PAL box with 50+ movies and was wondering what that weird thing is that people keep talking about.)

For all movies that I've seen, the forced captions (like elvish in lotr) is done by a normal selectable, default sub track, usually #1 english and forced display. Then there's of course the English for hearing impaired (usually one of the last tracks around #5-10).

Why dont they author the DVD's with regular forced display for R1? One reason I could think of is the same reason why they sell "full screen" versions of new movies for R1, stupid users who complain the subtitles dont show and that there's a fault in the picture since half of it is black. :D

Glad many of the HDTV's seem to be 16:9 so they dont have to pan&scan brand new 2.35:1 movies like it was VHS all over.

[cmsoliveira]

Hi,

being this my first post, I want to start by saying,thank very much, Jdobbs

Now, the reason for this post...

Using

dvd-rb 0.51b
cce 2.66.01.07
eclcce v1.7b
All options in dvd-rb as default except the One Pass vbr

The movie was "Joy ride", region 2, Pal

The result, which didn't happen due to low disk space (I had about 6GB free just for the rebuild fase), which means that the result was coming greatly oversized.

In the rebuilder.ecl shows this:

[item]
title=V02000300001004
aud_out=0
vaf_file=C:\ROADKILL\D2VAVS\V02000300001004.vaf
aud_file=C:\ROADKILL\D2VAVS\V02000300001004.mpa
file_focused=0
packet_size=2048
width=720
height=576
frame_rate_idx=3
cbr_brate=6000
vbr_brate_avg=4085
vbr_brate_min=0
vbr_brate_max=9000
seq_endcode=0
dvd=0
half_width=0
half_height=0
lum_level=0
aspect_ratio=3
gop_m=3
gop_nm=4
gop_hdr=12
seq_hdr=1
all_closed_gop=0
fix_gop_length=0
samples_per_sec=44100
stereo=2
brate_idx=7
crc=1
progressive=1
alternate_scan=0
intra_dc_prec=2
aud_mode=0
tc_ref_frm=0
drop_frame=0
fix_vbv_delay=0
letter_box=0
pulldown_detect=0
offset_line=0
create_new_vaf=1
credits_tweak=0
credits_start=0x00000
credits_brate=1000
h_filter=0
h_filter_idx=8
dither=0
dither_max=8
qmat_idx=0
quality_prec=16
video_type=2
vid_file0=C:\ROADKILL\D2VAVS\V02000300001004.m2v
vid_file1=C:\ROADKILL\D2VAVS\V02000300001004.m2v
vid_out=1
vaf_out=1
timecode=0x0000000
opv_q_factor=1
opv_brate_min=0
opv_brate_max=9000
vbr_bias=25
vbr_pass=1
use_filter=0
filter_val=6
non_linear=1
top_first=0
mpeg1=0
mpeg1_cps=1

[file]
name=C:\ROADKILL\D2VAVS\V02000300001004.avs
frame_first=0
frame_last=5788
encode_first=0
encode_last=5788


I believe that the error is in the opv_brate, but i'm just guessing...

DVD-Rb is the greatest, like is creator:)

[quantum]

Originally posted by Joergen
[B]Slightly OT: Well the closed captions problem(?) is not a problem for pal users since the CC feature is not used (I've never seen it mentioned on a PAL box with 50+ movies and was wondering what that weird thing is that people keep talking about.) Maybe it doesn't exist in PAL land. I don't know myself, but maybe you're not looking in the right place.

Many people don't know the difference. Subtitles are on the DVD and activated by the DVD remote. Closed captions (CC) are in the video stream and activated by the TV remote. CC works on regular television programs and VCR tapes as well as DVD.

Why would you care to have CC on a DVD when you usually have subtitles? The answer is a number of disks, particularly episodic TV disks like Farscape and plenty of others don't have subtitles at all, but they do have CC. So passing these through DVD Rebuilder loses the CC. DVD Shrink and the manual CCE method retains CC.

[jdobbs]

How does the manual CCE method keep CC?

[jdobbs]

@cmsoliveira

Welcome to the forum!

Yes this certainly seems to be in error, I can't image a time that Q would be set to 1. How big is the original disc?

[cmsoliveira]

Thx for the welcome, jdobbs :)

The disk (movie and extras) is about 6 Gb, no reauthoring, and decrypted in file mode with DVD decrypter 3.2.1.0

All the items appear with an opv_q_factor=1, for the extras and the main movie, only changing the opv_brate_max between 7200 and 9000

[quantum]

Originally posted by jdobbs
How does the manual CCE method keep CC?
You would have to ask that :(

I've done this before, but it's been a long time, so I wouldn't bet the house on this being accurate:

It starts with vsrip.exe which is part of the vobsub package:
http://www.doom9.org/Soft21/Subtitles/VobSub_2.23.exe

vsrip demuxes subtitles and also closed captions from title sets using an IFO as a parameter. I think it generates a RAW file.

Then you use vobsub2scc.exe to generate a SCC file which can be input as line21 in Scenarist. The line21 file in Scenarist is what causes the closed captions to be authored on the DVD.

You can get vobsub2scc.exe here and probably other places too:
http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_TOOLS.HTML

This thread may be of interest to you:
http://forum.doom9.org/showthread.php?s=&threadid=32646

I've done this many times before and I can get more detailed if it would be helpful to you. Let me know if there's anything I can do to help.

[DocDragon]

Originally posted by quantum
[...]
Why would you care to have CC on a DVD when you usually have subtitles? The answer is a number of disks, particularly episodic TV disks like Farscape and plenty of others don't have subtitles at all, but they do have CC. So passing these through DVD Rebuilder loses the CC. DVD Shrink and the manual CCE method retains CC.

:eek: quantum, thanks a lot for pointing that out! darn, i didn't pay attention to this little detail! looks like i need to redo some farscape episodes... :(

DD

[jdobbs]

Originally posted by quantum
You would have to ask that :(

I've done this before, but it's been a long time, so I wouldn't bet the house on this being accurate:

It starts with vsrip.exe which is part of the vobsub package:
http://www.doom9.org/Soft21/Subtitles/VobSub_2.23.exe

vsrip demuxes subtitles and also closed captions from title sets using an IFO as a parameter. I think it generates a RAW file.

Then you use vobsub2scc.exe to generate a SCC file which can be input as line21 in Scenarist. The line21 file in Scenarist is what causes the closed captions to be authored on the DVD.

You can get vobsub2scc.exe here and probably other places too:
http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_TOOLS.HTML

This thread may be of interest to you:
http://forum.doom9.org/showthread.php?s=&threadid=32646

I've done this many times before and I can get more detailed if it would be helpful to you. Let me know if there's anything I can do to help. I was afraid of that. If Scenarist did it, then I'd have to...:(


EDIT: Thanks for the references... I may be able to do this yet...

[Momotte]

jdobbs: Just tried LOTR3 with 0.51b OPVbr and looks fine when I keep everything (english/french ac3 tracks) tomorrow will try removing the french track.
(it tried q=55, q=54 and picked q=54 which seems right to me - I havn't tried encoding but I would guess everything is fine)

Maybe 0.51 has fixed my problem with OPV?! Will let you know tomorrow... time to go to bed.

[jdobbs]

Originally posted by quantum
@jdobbs
It seems the one-pass feature is number one on your to-do list. Any idea where multi-angle is? How about captions or the sub-title issue?

It seems to me the one-pass feature is interesting, but we can already finish an encode the traditional way albeit with more processing time, so it doesn't add anything we couldn't do before, or fix anything that was broken.

Of course it's up to you and I know some things are more "fun" to work on than others. Thought you might want to know -- I think I may have found something related to the subtitles. I don't know if it will definitely fix it, since I can't repeat the error, but I did find a difference between DVD-RB's output and that of DVD ReMake, which I've been told can correct the error with a no-change pass through the file... hopefully it will fix it. I certainly pray it does, as I think I've gone off the deep end in chasing this bug. You should see the modification posted tomorrow. I'll keep my fingers crossed.

[quantum]

Originally posted by jdobbs
something related to the subtitles. I don't know if it will definitely fix it, since I can't repeat the error, but I did find a difference between DVD-RB's output and that of DVD ReMake, which I've Good news. I know a programming problem is mentally difficult because you usually have no idea how to fix it, but it does seem to get fixed if you throw enough hours of frustration at it.

Bear in mind you have a finite number of problems to tackle that are reducing. One day you'll have all the problems fixed, then you'll be surprised when you realize you having nothing urgent left to work on.

[luphy]

Sorry if this is a question with an obvious answer, but I couldn't find anything relating to it:

When using OPV, you encode the movie at a certain Q-factor, which may or may not come out to the predicted size. With regards to the actual encoding, is CCE doing the exact same thing to the source material as it would with the regular mutipass method?

In other words, if given two encodes on the same source, one OPV, and the other the regular multipass VBR, and assuming (for this discussion) both end up having the same size and Q-factor - has CCE created identical outputs? Or more subjectively, would you say the two outputs are visually identical?

Call me cynical, but to be able to get the same size (roughly) output using OPV in much less time and with the same quality as would have been obtained using multipass.....seems too good to be true. :)

(I just did a movie only with English audio and subs.....original was 5.67 gb, and DVD-RB output (4.26 gb) at Q of 11....encode phase only took 73 minutes - only very slightly longer than a transcode! A transcode probably would have looked fine, considering the selected Q was only 11 :) )

[jdobbs]

Heres a quick summary of the difference:

In multiple pass VBR, the encoder first goes over the entire input stream and does a mock encode (or sometimes an actual encode) at a certain bitrate. It then can get a good idea of the level of quantization it needs to apply to different areas of the source. It then balances the available bitrate across the source (within the minimum and maximums provided) so you get the most consistent quality while staying within the required average bitrate. It will very accurately predict the output size (almost exactly).

In one pass VBR (in general not any specific brand-name or implementation) the encoder fixes it's quantization levels at some predetermined value (which generally predicts quality) and uses whatever bitrate is needed (within the set limits) to reach it. That way you can be sure your quality is consistent across the source -- but you can't predict output size.

What happens in the OPV implementation in DVD-RB is borrowed from RoBa techniques. A Q value is selected, and a sample (some percentage of the whole source, e.g. 1%) is encoded and the size is compared to what is desired. Then a prediction is made as to what Q would actually get you within the size limits. This serves the same purpose as the first pass of a multiple pass VBR in much less time. It can give you close to (if not equal to) the same quality.

The disadvantage to OPV is that sometimes the sample isn't representative enough of the whole source, so the size can be off. If it goes over -- you have to take measures (like an additional encoding pass, or requantization) to fix it. But it is pretty good, and tylo has given me some good tips on how to make it fairly accurate.

The second disadvantage is that you have to use the SP version of CCE to take advantage of it... and that is an expensive proposition generally limited to AV professionals.

[luphy]

I just watched my OPV encode of The Last of the Mohicans.

As mentioned previously the encode took around 73 minutes. Now I did run the movie through Shrink to select movie only, uncompressed (did not remove any subtitles or audios), so it may be related to Shrink, but I've never had any problems with Shrink uncompressed material affecting DVD-RB.

At the end of encoding (in OPV and regualr 2-pass, I get the error message at the end of the encode that old VOB number was 1, and new was 3 - or something to that extent....what does that mean?).

The OPV encode plays fine on my WinDVD, but when played from my standalone, at multiple scenes (low action and high action), it would jump forward a whole section of the movie, usually to the next chapter. If I were to fast forward through that one part where it jumps, it will then play the segment fine. Again, there were numerous instances of this...not sure if it's related to that error message. It wasn't a burning issue because I burned it twice (different disc), and both times the jumps occur at the exact same places.

I reran the same source (Shrink output) through 2-pass, and so far the movie plays fine on my standalone.

Not sure if it's related to method of encoding or not...??