BD Rebuilder Beta - Bug Reports Only

[jdobbs]

I'm curious if there will be a time difference between LAVF & multiprocessing. If it works out well I'm hoping to add a module to split for LAVF as well. I'm also doing some testing with X264's internal resizing too -- that way LAVF mode could handle some of the ALTERNATE formats that require resizing -- and even use the 64 bit routines to do it. With no filters, LAVF mode should be able to do pretty much anything as long as you aren't trying to process a VC-1 interlaced source.

I think the program has gotten to a point where it is stable (for the most part, as long as the helper apps don't get hosed by some other process) -- so I'm now looking at ways to improve speeds.

[colinhunt]

Unfortunately I can't change the number of X264 instances between passes 1 and 2 -- because X264 needs them to be identical. It would also require another split -- which would also add time.
Very likely a stupid question as I'm not a programmer... So it's not possible to set instances to 4 for pass 1, and then during pass 2 drop instances to 2 while tellling the instances to process parts sequentially? For example, instance #1 begins processing part 1 and #2 begins with part 2. Once #1 is done with part 1 it begins processing part 3 and #2 moves to part 4 once its done with part 2.

Or how about starting 4 instances but only giving 2 instances parts to process? Too much of a kludge? ;)

I suppose further means of improving speed would include moving to 64-bit apps and multithreading, like Avisynth-MT.

[Ch3vr0n]

multiprocessing LAVF would be cool to see happening. multiprocessing LAFV x264-64 would be even better. That would definitely give a major speed increase. 80% into the current build and ram usage per instance is holding steady arround 800MB each, total cpu usage at 100%, total ram use at 5.2GB

** edit: recoding for main m2ts complete: job time including splitting and rejoining multiparts 93minutes**

i did notice some weird behavior. LAVF was disabled in the setup. yet x264-64 was launched and cpu usage was approx 100% for 1 pass encodes on the secondairy files. According to the release notes from September 5th, 2010 x264-64 should only be launched when LAVF is selected on a 64bit os. My OS is 64bit but lavf was not selected.

[Status]
LABEL=FRIENDS_WITH_BENEFITS
VERSION=v0.40.01 (beta)
SOURCE_SIZE=36453851971
SOURCE_VIDEO_SIZE=35969003520
TARGET_SIZE=24641536000
REDUCTION=.671597352858776
RESIZE_1080=0
AUDIO_TO_KEEP=dut;eng;nld;und;
KEEP_HD_AUDIO=-1
SUBS_TO_KEEP=dut;eng;nld;und;
BACKUP_MODE=0
MOVIEONLY_TYPE=0
USE_LAVF=0
MULTIPROCESS=3
QUICK=0
ENCODE_STEP=1.5
COMPLETED=6

** edit 2: well this is even weider. Now x264-64 doenst get loaded on VID_00102. On some it does others it doesnt. Weird"
** edit 3: Iso burning completed at 17.53. Job started on 14.45. So 2hrs for an 125video encoding and 3 blanks" Total size output 23.364.096kb aka 22.2GB
Now redoing same job with LAVF enabled. To check time diff

Job Restarted at 18.00.00hrs

[jdobbs]

multiprocessing LAVF would be cool to see happening. multiprocessing LAFV x264-64 would be even better. That would definitely give a major speed increase. 80% into the current build and ram usage per instance is holding steady arround 800MB each, total cpu usage at 100%, total ram use at 5.2GB

** edit: recoding for main m2ts complete: job time including splitting and rejoining multiparts 93minutes**

i did notice some weird behavior. LAVF was disabled in the setup. yet x264-64 was launched and cpu usage was approx 100% for 1 pass encodes on the secondairy files. According to the release notes from September 5th, 2010 x264-64 should only be launched when LAVF is selected on a 64bit os. My OS is 64bit but lavf was not selected.

** edit 2: well this is even weider. Now x264-64 doenst get loaded on VID_00102. On some it does others it doesnt. Weird" X264-64 might also be loaded for segments that have ATCDelta set. That was implemented because of a glitch in certain instances related to framecounts processed on multipart sources. I'll look at it and see if it is still necessary with the splitting...

[varekai]

Thought I'd share this, I'm a not-so-settings-savvy-person.
I just fired up BDRB like I use to do.
Usually I select ENCODE_QUALITY=4 and let it run over-nite.
But for this test I lowered the quality.

The new version 0.40.01 is a lot faster! :D

-----------------------
[11:49:38] BD Rebuilder v0.39.07 (beta)
- Source: xxx_xxx_xxx_x
- Input BD size: 43,16 GB
- Approximate total content: [02:48:21.091]
- Target BD size: 23,05 GB
- Windows Version: 6.1 [7601]
- MOVIE-ONLY mode enabled
- Quality: Good (Very Fast), Two Pass
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
[11:49:38] PHASE ONE, Encoding
- [11:49:38] Processing: VID_00050 (1 of 1)
- [11:49:38] Extracting A/V streams [VID_00050]
- [11:56:03] Reencoding video [VID_00050]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23,976fps, 242*184 frames
- Bitrate: 13*606 Kbs
- [11:56:03] Reencoding: VID_00050, Pass 1 of 2
- [13:29:56] Reencoding: VID_00050, Pass 2 of 2
- [15:14:36] Video Encode complete
- [15:14:37] Processing audio tracks
- Track 4352 (xxx): Keeping original audio
- [15:14:37] Multiplexing M2TS
[15:20:30]PHASE ONE complete
[15:20:30]PHASE TWO - Rebuild Started
- [15:20:30] Rebuilding BD file Structure
[15:29:24] - Encode and Rebuild complete
[15:29:24]JOB: xxx_xxx_xxx_x finished.
-----------------------

-----------------------
[15:38:40] BD Rebuilder v0.40.01 (beta)
- Source: xxx_xxx_xxx_x
- Input BD size: 43,16 GB
- Approximate total content: [02:48:21.091]
- Target BD size: 23,05 GB
- Windows Version: 6.1 [7601]
- MOVIE-ONLY mode enabled
- Quality: Good (Very Fast), Two Pass
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
[15:38:40] PHASE ONE, Encoding
- [15:38:40] Processing: VID_00050 (1 of 1)
- [15:38:40] Extracting A/V streams [VID_00050]
- [15:45:13] Reencoding video [VID_00050]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23,976fps, 242*184 frames
- Bitrate: 13*606 Kbs
- [15:45:13] Reencoding: VID_00050, Pass 1 of 2
- [16:29:37] Reencoding: VID_00050, Pass 2 of 2
- [17:13:58] Video Encode complete
- [17:13:59] Processing audio tracks
- Track 4352 (xxx): Keeping original audio
- [17:13:59] Multiplexing M2TS
[17:20:06]PHASE ONE complete
[17:20:06]PHASE TWO - Rebuild Started
- [17:20:06] Rebuilding BD file Structure
[17:29:10] - Encode and Rebuild complete
[17:29:10] JOB: xxx_xxx_xxx_x finished.
-----------------------

[jdobbs]

Thought I'd share this, I'm a not-so-settings-savvy-person.
I just fired up BDRB like I use to do.
Usually I select ENCODE_QUALITY=4 and let it run over-nite.
But for this test I lowered the quality.

The new version 0.40.01 is a lot faster! :D
Did you have MULTIPROCESS set?

[varekai]

Did you have MULTIPROCESS set?
No, not set.
I havn't looked into the HIDDENOPTS.TXT until now.
Will read more about it on your site.
I can do some more testings if you like?

[Rumbah]

The new x264 is much faster for me, too. 10-20% without any setting changed in some test encodes of mine (not tested with bdrb)

[colinhunt]

Now that one of the three x264 instances finished its job and the process is running two instances only, FPS went from ~17 to ~18.5 (now actually 19.00, whee! ... less than 10% to go and fps is now 19.50.). And while total CPU usage is still at 100%, the PC is much more responsive to use than it was when 3 instances were running.

It will be interesting to see how many instances BD-RB will launch when I set multiprocess to automatic (i.e. 1). On my PC 4 instances is optimal for Pass 1 (as extrapolated from current data) but 2 instances works best for Pass 2.

[Ch3vr0n]

@Jdobbs:

2nd rebuild of Friends with Benefits completed, LAVF enabled

Job 2: LAVF (multithread enabled, though according to you not used in LAVF situations)
Job Start: 18.00hrs
Job completion: 20.26hrs (including iso burning)
Total duration: approx 2.5hrs

Job 1: multithread enabled
Job Start: 14.45
Job Completion: 17.53 (including iso burning)
Total Duration: approx 3hrs 10 minutes

Total timeLOSS with multi thread (including splitting & rejoining) 40 minutes unless my math is off

I'd say that's a pretty significant timeloss for an experimental feature.

[colinhunt]

BD-RB just finished Pass 2 of a 290,138 frame main movie. Based on experience I can estimate previous versions of BD-RB would have taken approx. 400-450 minutes to process that Pass 2. This new version of BD-RB ran Pass 2 in 296 minutes with multiprocess enabled... and it would have been even faster if it had been running only 2 instances of x264 from the start instead of 3. Pass 1 took 100 minutes and I'm pretty certain earlier versions would have taken at least twice as long, perhaps more.

I'd say the total time saved is something like 3-4 hours. You know what that is? Awesome, that's what.

[soneca]

1 instance

- [15:33:42] Reencoding: VID_00000, Pass 1 of 2
- [15:58:33] Reencoding: VID_00000, Pass 2 of 2
- [17:18:10] Video Encode complete

3 instances
ram max pass 1 ~ 4,05GB
ram max pass 2 ~ 3,65GB
- [13:44:10] Reencoding: VID_00000, Pass 1 of 2
- [14:03:00] Reencoding: VID_00000, Pass 2 of 2
- [15:22:17] Video Encode complete

4 instances
ram max pass 1 ~ 5,06GB
ram max pass 2 ~ 4,5GB
- [10:40:58] Reencoding: VID_00000, Pass 1 of 2
- [10:59:39] Reencoding: VID_00000, Pass 2 of 2
- [12:19:40] Video Encode complete

I believe that using two instances in my case would be a good balance between speed on the first pass and consumption of ram.

[colinhunt]

Running the same backup again on v0.39.07 to see what sort of FPS figures I get.

Source: AVC/480i (DEINT processing enabled)
PASS 1
v0.39.07 - 179 fps
v0.40.01 - 381 fps (3 instances)

PASS2
v0.39.07 - 100 fps
v0.40.01 - 117 fps (3 instances)

from ~75% completion onwards:
v0.39.07 - 92 fps
v0.40.01 - 133 fps (2 instances)

Source: AVC/1080p23.976
PASS 1
v0.39.07 - 30 fps (...and dropping)
v0.40.01 - 63 fps (3 instances)

PASS 2
at 10% completion:
v0.39.07 - 13 fps (higher than expected!)
v0.40.01 - 17 fps (3 instances)


I user bd-rebuilder.log to compare how long it took for 0.39.07 and 0.40.01 to make two passes of the same main movie file using the exact same settings.

v0.40.01: 6 hours 36 minutes
v0.39.07: 7 hours 54 minutes

Despite higher fps figures especially for Pass 1, the difference in total time is not quite as big as I originally estimated. I was expecting to see 11-12 fps for Pass 2 from v0.39.07 but the disc I picked for the test was "easier" to re-encode than I thought it would be.

[jdobbs]

@Jdobbs:

2nd rebuild of Friends with Benefits completed, LAVF enabled

Job 2: LAVF (multithread enabled, though according to you not used in LAVF situations)
Job Start: 18.00hrs
Job completion: 20.26hrs (including iso burning)
Total duration: approx 2.5hrs

Job 1: multithread enabled
Job Start: 14.45
Job Completion: 17.53 (including iso burning)
Total Duration: approx 3hrs 10 minutes

Total timeLOSS with multi thread (including splitting & rejoining) 40 minutes unless my math is off

I'd say that's a pretty significant timeloss for an experimental feature. Gotta be something wrong there. At the most you should lose no more than the time associated with splitting... and in most cases that would be 5-10 minutes. The only other possibility might be if you are running short on memory.

[jdobbs]

No, not set.
I havn't looked into the HIDDENOPTS.TXT until now.
Will read more about it on your site.
I can do some more testings if you like? If you're getting improvements without MULTIPROCESS set I would have to guess the improvement is in the new version of X264. Have you checked the output and it looks good/normal?

[jdobbs]

The new x264 is much faster for me, too. 10-20% without any setting changed in some test encodes of mine (not tested with bdrb) That's good to know. I'll go back through the changelog of X264 and see where it could pick up that much. 10-20% seems likea lot.

[soneca]

The new x264 is much faster for me, too. 10-20% without any setting changed in some test encodes of mine (not tested with bdrb)
I got a gain of ~12% using the 64bit version and preset slower.

[Rumbah]

That's good to know. I'll go back through the changelog of X264 and see where it could pick up that much. 10-20% seems likea lot.

There were CABAC Trellis asm optimizations. Must have been the last commits.

[Ch3vr0n]

Gotta be something wrong there. At the most you should lose no more than the time associated with splitting... and in most cases that would be 5-10 minutes. The only other possibility might be if you are running short on memory.

Doubt it, never ran above 5.5GB. Got 8 gig installed. I'll run another encode tomorrow. Its a bit too late now.

[dfsooner]

I set multiprocess=3 but I'm only seeing one instance of x264 and it is 32-bit. What do I need to do to use multiple instances of 64-bit x264?

I'm seeing the desired increase in CPU usage but only modest memory usage increase.

Intel i7 2600K - 8GB Windows 7 x64.

[Sharc]

From a BD Rebuilder point-of-view there is no difference between a BD-5 and a BD-9.

I just finished "Rise of the Planet of the Apes" from the BD-9 backup win ALTERNATE_PAL=1. It also was converted to AC3 5.1 from the original. It has perfect sync throughout. It's possible you may have a CODEC issue... but don't know for sure. I'd suggest you uninstall/reinstall BD-RB, HAALI, AVISYNTH, and FFDSHOW.
My latest findings:
Sync is ok with ALTERNATE_PAL=0 (but then I get the ugly blends)
Sync fails with ALTERNATE_PAL =1.
(I have reinstalled everything but Avisynth. This may be my next trial).

[bassnut]

just redooing my oc now and will run some tests to with my 1090T tomorrow once i have tested for stability.

[jdobbs]

I set multiprocess=3 but I'm only seeing one instance of x264 and it is 32-bit. What do I need to do to use multiple instances of 64-bit x264?

I'm seeing the desired increase in CPU usage but only modest memory usage increase.

Intel i7 2600K - 8GB Windows 7 x64. Reasons that could happen:

1. LAVF is set.
2. Resizing. (ALTERNATE resizing or selecting one of the resize options in SETUP)
3. VC-1 Interlaced source.

[setarip_old]

@jdobbs

If, "- Corrected an issue related to the program map and remuxing of DTS Express and IGS streams." means that all standalone players should now properly process ExpressDTS audio, I'm sorry to tell you that I used v.40.01 to produce a 25Gb disc of one of the "Band of Brothers" Blu-rays - and playback on my SONY BDP S360 exhibited the same old behavior of silent PIP inserted video, with reduced volume primary audio.

If the "corrected issue" was NOT expected to accomplish this, please advise.

[jdobbs]

@jdobbs

If, "- Corrected an issue related to the program map and remuxing of DTS Express and IGS streams." means that all standalone players should now properly process ExpressDTS audio, I'm sorry to tell you that I used v.40.01 to produce a 25Gb disc of one of the "Band of Brothers" Blu-rays - and playback on my SONY BDP S360 exhibited the same old behavior of silent PIP inserted video, with reduced volume primary audio.

If the "corrected issue" was NOT expected to accomplish this, please advise. Nope. It didn't mean that at all. This was an unrelated issue that also had to be fixed.

[setarip_old]

Considering my results, I'm glad to hear that ;>}

[omegaman7]

Clearly I need a more powerful system to benefit from the new code. I did hear you say (Jdobbs), that you didn't benefit much. Although perhaps playing with more settings. I'm pretty sure this version is friskier though. Apparently the new X264 has better optimizations. In any case, my results from a star trek VI encode.


All done at "High Quality"
Main title encoding
1hr:04m:35s 1st pass (MULTIPROCESS=1)
2hr:31m:11s 2nd pass (MULTIPROCESS=1)


1hr:05m:26s 1st pass (MULTIPROCESS=DISABLED)
2hr:29m:28s 2nd pass (MULTIPROCESS=DISABLED)

Overall encode
4hr:24m:47s (MULTIPROCESS=1)
4hr:21m:38s (MULTIPROCESS=DISABLED)

[dfsooner]

Reasons that could happen:

1. LAVF is set.
2. Resizing. (ALTERNATE resizing or selecting one of the resize options in SETUP)
3. VC-1 Interlaced source.

Source is MPEG-4 AVC 1080p
LAVF not set.
INF file:
[Status]
LABEL=CHUCK_S1D3
VERSION=v0.40.01 (beta)
SOURCE_SIZE=19593642692
SOURCE_VIDEO_SIZE=19558010880
TARGET_SIZE=8422162432
REDUCTION=.428802840506447
RESIZE_1080=0
AUDIO_TO_KEEP=eng;
KEEP_HD_AUDIO=-1
SUBS_TO_KEEP=eng;fra;ger;spa;
BACKUP_MODE=0
MOVIEONLY_TYPE=0
USE_LAVF=0
MULTIPROCESS=1
QUICK=0
ENCODE_STEP=1.5
REBUILD_COMPLETE=0
[00001]
AUDIO=100
PGS=1110000
VIDEO2=0
V2MBRATE=0
M2TS_TARGET=2412705328
RATE=6175

Also noticed that MULTIPROCESS=1 is inserted even though I specified a value of 3 in the .ini file.

Can x264-64 be specified or does the the program choose the 32-bit or 64-bit dynamically?

[Chuckwagon]

Just adding my results so far testing the new multiprocessor code.

Adding MULTIPROCESS=n to the Config file seems to work, as the log file reports MULTIPROCESS=3 when I set it to MULTIPROCESS=3 in the Config file, so I am assuming it is doing as it should.

My system is an i7 X980 with 6GB of RAM. I am doing a full backup of X-Men: First Class.
- Input BD size: 44.11 GB
- Approximate total content: [04:35:20.959]
- Target BD size: 22.95 GB
- Windows Version: 6.1 [7601]
- Quality: High-Speed Option (BD-25), ABR
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640

I used the times from "PHASE ONE, Encoding" and "Encode and Rebuild complete" as reported in the log file as my start and stop times to calculate the total time for the backup. (I left out the time it takes to write the ISO just in case there are any size differences or hard drive hiccups.)

With MULTIPROCESS=0 the total time is 1:29:22, and the completed ISO is 23,001,024 KB in size.

With MULTIPROCESS=1 the total time is 1:29:20, and the completed ISO is 23,001,088 KB in size.

With MULTIPROCESS=3 the total time is 1:28:02, and the completed ISO is 22,994,240 KB in size.

So, for my system, with this disc, there is very little change with the various settings.

I will try again another disc and report again.

[Yordan5]

Intel Core i5 2500K, 6Gb RAM, Windows 7 64bit.

Input file (bluray movie only) 26Gb

Output, bluray DVD5, 1080p, audio downmix to Stereo 192khz

MULTIPROCESS=3

During encoding Pass 1 and Pass 2 three instances of x264 were running.

From start to end it took 3.08hrs to complete the whole movie.

Then I ran the same settings again (just to compare if MULTIPROCESS would make any difference) only this time I disabled MULTIPROCESS. However, I noticed that for some reason BD Rebuilder ran MULTIPROCESS=1 (even though I disabled it by removing the MULTIPROCESS line from the edit list). During encoding Pass 1 only one instance of x264 was running. I did not stay around to see if it would be different during Pass 2.

From Start to end it took about 8min longer this time.

[colinhunt]

I'm running the same disc (Malcolm X, if you must know) again with 0.40.01, but this time with multiprocess=1... and BD-RB is splitting the main movie file for 10-way encoding. Whoa! This should be interesting ;)

** update
Pass 1 is reaching 84 fps on 10 instances. On 3 instances (set manually) it reached 63 fps.
Oh wow, total RAM use 11.8 GB. Of course; 10 instances eating more than 1GB each.

I'm pretty sure 10 instances will choke on Pass 2. We'll see.

** update
The system started running out of RAM halfway through Pass 1. Swapping memory to HDD caused an instant performance hit and the PC became extremely sluggish to use.

** update
Uh-huh. It's getting pretty bad. I don't like the sounds my encoding PC is making. I think I have to kill BD-RB process. With extreme prejudice. Kill it with fire.

** update
It's done. What a gruesome scene. I barely managed to make it out alive. Let this be a cautionary tale to one and all.

[colinhunt]

However, I noticed that for some reason BD Rebuilder ran MULTIPROCESS=1 (even though I disabled it by removing the MULTIPROCESS line from the edit list). During encoding Pass 1 only one instance of x264 was running
It's not multiprocessing if it's running only a single instance.

[bassnut]

Running CPLOMBIANA now will see what the results are when I get home.

Just a note to Norton Internet Security 2012 users ...... Nortons is being a pain again and is being dificult even when adding BDREBUILDER to exclusions.

[varekai]

Did you have MULTIPROCESS set?
If you're getting improvements without MULTIPROCESS set I would have to guess the improvement is in the new version of X264. Have you checked the output and it looks good/normal?
Tried new settings.
USE_LAVF=0
MULTIPROCESS=1

CPU load on reencoding 90-100% (i7 2600K @ 4.3GHz)
No memory issues, stable at 10% (16GB)

Quality: Good (Very Fast), Two Pass encodings look great on PC display.
Will burn to a BD-RE and check with Oppo and plasma.
Output looks alright though a higher encode quality will improve for shure.
Next test is to enable ENCODE_QUALITY=4

-----------------------
[09:38:44] BD Rebuilder v0.40.01 (beta)
- Source: xxx_xxx_xxx_x
- Input BD size: 43,16 GB
- Approximate total content: [02:48:21.091]
- Target BD size: 23,05 GB
- Windows Version: 6.1 [7601]
- MOVIE-ONLY mode enabled
- Quality: Good (Very Fast), Two Pass
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
[09:38:44] PHASE ONE, Encoding
- [09:38:44] Processing: VID_00050 (1 of 1)
- [09:38:44] Extracting A/V streams [VID_00050]
- [09:45:17] Reencoding video [VID_00050]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23,976fps, 242*184 frames
- Bitrate: 13*606 Kbs
- [09:45:17] Reencoding: VID_00050, Pass 1 of 2
- [10:29:42] Reencoding: VID_00050, Pass 2 of 2
- [11:14:01] Video Encode complete
- [11:14:02] Processing audio tracks
- Track 4352 (xxx): Keeping original audio
- [11:14:02] Multiplexing M2TS
[11:20:35]PHASE ONE complete
[11:20:35]PHASE TWO - Rebuild Started
- [11:20:35] Rebuilding BD file Structure
[11:29:39] - Encode and Rebuild complete
[11:29:39] JOB: xxx_xxx_xxx_x finished.
-----------------------

[Yordan5]

It's not multiprocessing if it's running only a single instance.

I understand that. MULTIPROCESS=1 means that BD Rebuilder will decide if additional instances are required (looks like here it decided multiple instances were not required). I only mentioned it as I specifically deleted MULTIPROCESS from the Edit list and yet it appeared again.

[steveg32]

@jdobbs

"Error in attempt to multiplex: MUX_00136.meta
- Can't open file: ... WORKFILES\VID_00136_2.AVS.MKV"

--------------------------------

- Windows Version: 6.1 [7601]
- AVISYNTH Version: [2.5.8.0], Ok
- HAALI Splitter: [1.9.42.1], Ok
- FFDSHOW: [3882], Ok
- WIN7 preferred AVC CODEC: Ok
- WIN7 preferred VC-1 CODEC: Ok
- WIN7 preferred MPEG2 CODEC: Ok
- FFDSHOW VC-1 set to "wmv9", Ok
- FFDSHOW MPEG2 set to "libavcodec": Ok
- FFDSHOW AVC set to "ffmpeg-mt": Ok
- BD Rebuilder v0.40.0.1, Ok
- X264: Ok
- AFTEN: Ok
- FAAC: Ok
- MP4BOX: Ok
- WAVI: Ok
- TSMUXER: Ok

--------------------------

[Options]
VERSION=0.40.0.1
DTSX_ENABLE=1
MODE=0
ENCODE_QUALITY=0
ONEPASS_ENCODING=2
AUTO_QUALITY=1
MINIMIZE_TO_TRAY=1
TARGET_SIZE=23500
AUTO_BURN=2
AUDIO_TO_KEEP=eng;
SUBS_TO_KEEP=all
SD_CONVERT=0
OPEN_GOP=0
RESIZE_1080=0
DEINTERLACE=1
SD_TO_1080=0
CONVERT_WIDE=0
DTS_REENCODE=0
AC3_REENCODE=0
AC3_640=1
AC3_192=0
KEEP_HD_AUDIO=1
AVCHD=1
REMOVE_WORKFILES=1
MOVIE_ONLY_LOOP=1
REMOVE_OUTPUT=0
USE_FILTERS=0
BDMV_CERT_ONLY=1
USE_LAVF=0
IVTC_PULLDOWN=0
ASSUME_DVD_PAL=0
UNMASK_CHAPTER=0
COMPLETION_BEEP=1
AUDIO_TRACK_LIMIT=0
SUBTITLE_TRACK_LIMIT=0
CUSTOM_TARGET_SIZE=23500
QUICK_EXTRAS=1
[Paths]
SOURCE_PATH=C:\USERS\STEVE\VIDEOS\MY ANYDVD HD BLU-RAY RIPS\
WITH DT EXPRESS AUDIO\BATTLE_LA\WORKING_PATH=C:\USERS\STEVE\
VIDEOS\MY BD-REBUILDER ENCODES\MY BD-25_GOOD_1 PASS ABR\

--------------------

[02:12:52] BD Rebuilder v0.40.01 (beta)
- Source: BATTLE_LA
- Input BD size: 39.58 GB
- Approximate total content: [03:52:52.748]
- Target BD size: 22.95 GB
- Windows Version: 6.1 [7601]
- Auto Quality: Good (Very Fast), ABR
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
[02:12:52] PHASE ONE, Encoding
- [02:12:52] Processing: VID_00127 (1 of 34)
- [02:12:52] Extracting A/V streams [VID_00127]
- [02:12:55] Reencoding video [VID_00127]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 165 frames
- [02:12:55] Reencoding: VID_00127, Pass 1 of 1
- [02:12:57] Video Encode complete
- [02:12:57] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- [02:12:57] Multiplexing M2TS
- [02:13:01] Processing: VID_00101 (2 of 34)
- [02:13:01] Extracting A/V streams [VID_00101]
- [02:13:08] Reencoding video [VID_00101]
- Source Video: MPEG-2, 1920x1080
- Rate/Length: 23.976fps, 1,012 frames
- [02:13:08] Reencoding: VID_00101, Pass 1 of 1
- [02:13:19] Video Encode complete
- [02:13:19] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- [02:13:19] Multiplexing M2TS
- [02:13:23] Processing: VID_00104 (3 of 34)
- [02:13:23] Extracting A/V streams [VID_00104]
- [02:13:30] Reencoding video [VID_00104]
- Source Video: MPEG-2, 1920x1080
- Rate/Length: 23.976fps, 1,012 frames
- [02:13:30] Reencoding: VID_00104, Pass 1 of 1
- [02:13:41] Video Encode complete
- [02:13:41] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- [02:13:41] Multiplexing M2TS
- [02:13:45] Processing: VID_00129 (4 of 34)
- [02:13:45] Extracting A/V streams [VID_00129]
- [02:13:53] Reencoding video [VID_00129]
- Source Video: MPEG-2, 1920x1080
- Rate/Length: 23.976fps, 1,330 frames
- [02:13:53] Reencoding: VID_00129, Pass 1 of 1
- [02:14:07] Video Encode complete
- [02:14:07] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- [02:14:07] Multiplexing M2TS
- [02:14:12] Processing: VID_00116 (5 of 34)
- [02:14:12] Extracting A/V streams [VID_00116]
- [02:14:19] Reencoding video [VID_00116]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 1,883 frames
- [02:14:19] Reencoding: VID_00116, Pass 1 of 1
- [02:14:47] Video Encode complete
- [02:14:47] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- [02:14:47] Multiplexing M2TS
- [02:14:52] Processing: VID_00136 (6 of 34)
- [02:14:52] Extracting A/V streams [VID_00136]
- [02:15:00] Reencoding video [VID_00136]
- [02:15:00] Reencoding secondary video [TRK_02]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 856 frames
- Bitrate: 14,731 Kbs
- [02:15:02] Reencoding: VID_00136, Pass 1 of 1
- [02:15:17] Video Encode complete
- [02:15:17] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- Track 6656 (eng): Keeping original audio
- Track 4353 (eng): Keeping original audio
- [02:15:17] Multiplexing M2TS
- Error in attempt to multiplex: MUX_00136.meta
- Can't open file: C:\USERS\STEVE\VIDEOS\MY BD-REBUILDER ENCODES\MY
BD-25_GOOD_1 PASS ABR\WORKFILES\VID_00136_2.AVS.MKV
[02:15:21] - Failed to build structure, aborted

----------------

"DTSX_ENABLE=1" is the only change I added to my config/INI file. Tried 3x but always ends like this. Mahalo, Steve :confused:

Tried again but without "DTSX_ENABLE=1" added to my config/INI file. Got the exact error again. Mahalo, Steve :confused:

[jdobbs]

Intel Core i5 2500K, 6Gb RAM, Windows 7 64bit.

Input file (bluray movie only) 26Gb

Output, bluray DVD5, 1080p, audio downmix to Stereo 192khz

MULTIPROCESS=3

During encoding Pass 1 and Pass 2 three instances of x264 were running.

From start to end it took 3.08hrs to complete the whole movie.

Then I ran the same settings again (just to compare if MULTIPROCESS would make any difference) only this time I disabled MULTIPROCESS. However, I noticed that for some reason BD Rebuilder ran MULTIPROCESS=1 (even though I disabled it by removing the MULTIPROCESS line from the edit list). During encoding Pass 1 only one instance of x264 was running. I did not stay around to see if it would be different during Pass 2.

From Start to end it took about 8min longer this time. Don't confuse the value in the INF file with the one in the INI. The one in the INF says how many instances are actually running, the one in the INI tell BD-RB how to determine that number. In other words, setting it to "0" in the INI would mean "1" in the INF. Setting a "1" in the INI could mean any number in the INF depending upon your system.

I've changed the label in the INF to "INSTANCES" for the next release (just to avoid confusion).

[jdobbs]

I'm running the same disc (Malcolm X, if you must know) again with 0.40.01, but this time with multiprocess=1... and BD-RB is splitting the main movie file for 10-way encoding. Whoa! This should be interesting ;)

** update
Pass 1 is reaching 84 fps on 10 instances. On 3 instances (set manually) it reached 63 fps.
Oh wow, total RAM use 11.8 GB. Of course; 10 instances eating more than 1GB each.

I'm pretty sure 10 instances will choke on Pass 2. We'll see.

** update
The system started running out of RAM halfway through Pass 1. Swapping memory to HDD caused an instant performance hit and the PC became extremely sluggish to use.

** update
Uh-huh. It's getting pretty bad. I don't like the sounds my encoding PC is making. I think I have to kill BD-RB process. With extreme prejudice. Kill it with fire.

** update
It's done. What a gruesome scene. I barely managed to make it out alive. Let this be a cautionary tale to one and all. 10-way encoding with MULTIPROCESS=1? You'd have to have at least 16 processors (as indicated by the NUMBER_OF_PROCESSORS environment variable) and 12GB of memory for that to happen. The current algorithm gives you the lesser of (.6 x processors) or (memory / 1GB). I'll adjust it down for the next release.

What kind of monster system are you using? I'm guessing it is the kind for which this feature was actually intended. So with MULTIPROCESS=3 you're seeing about a 50% increase in encoding speed, apparently. Obviously 10 processes was a bad idea -- can you try it with 6 or so?

[jdobbs]

@steveg32

I'll do some testing and see if the new code has somehow affect the muxing of DTS-X.

[Chuckwagon]

I have completed a second comparison, only this time I used the "MOVIE and MENUS" mode on a disc (Diehard) that has one large main movie stream and not many smaller streams (X-Men: First Class.)

Using the same system, an i7 X980 with 6GB of RAM, and these settings,
- Input BD size: 29.61 GB
- Approximate total content: [03:00:38.394]
- Target BD size: 22.95 GB
- Windows Version: 6.1 [7601]
- MOVIE and MENUS mode enabled
- Quality: High-Speed Option (BD-25), ABR
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
I got these results:

With MULTIPROCESS=0 the total time is 1:21:53, and the completed ISO is 23,183,232 KB in size.

With MULTIPROCESS=3 the total time is 1:09:09, and the completed ISO is 23,256,320 KB in size.

So for this disc, using the mp setting at 3 results in an encode that is 12:44 faster than the single processor run, with an output ISO that is 73,088 KB larger.

So I've been able to see good improvement (15%) in encoding speed with one disc, but no change with another. The differences between them were full backup vs movie and menus only, and a main title comprised of many files vs just one.

Is there a minimum size an extracted file needs to be in order to be split for multiprocessing? If not, I will try a full backup of Diehard, and a movie and menus backup of X-Men: First Class, and see if either change alters the behavior.

[jdobbs]

@steveg32

Is MKV_INTERMEDIATE=1 set in your INI file?

[jdobbs]

I have completed a second comparison, only this time I used the "MOVIE and MENUS" mode on a disc (Diehard) that has one large main movie stream and not many smaller streams (X-Men: First Class.)

Using the same system, an i7 X980 with 6GB of RAM, and these settings,
- Input BD size: 29.61 GB
- Approximate total content: [03:00:38.394]
- Target BD size: 22.95 GB
- Windows Version: 6.1 [7601]
- MOVIE and MENUS mode enabled
- Quality: High-Speed Option (BD-25), ABR
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
I got these results:

With MULTIPROCESS=0 the total time is 1:21:53, and the completed ISO is 23,183,232 KB in size.

With MULTIPROCESS=3 the total time is 1:09:09, and the completed ISO is 23,256,320 KB in size.

So for this disc, using the mp setting at 3 results in an encode that is 12:44 faster than the single processor run, with an output ISO that is 73,088 KB larger.

So I've been able to see good improvement (15%) in encoding speed with one disc, but no change with another. The differences between them were full backup vs movie and menus only, and a main title comprised of many files vs just one.

Is there a minimum size an extracted file needs to be in order to be split for multiprocessing? If not, I will try a full backup of Diehard, and a movie and menus backup of X-Men: First Class, and see if either change alters the behavior. Yes there is a minimum size. BD-RB will not split a file unless there are a minimum of 5000 frames per segment. That's to ensure there isn't a quality drop because of a shortage of frame samples in which to distribute bitrate. That means that multi-part sources are less likely to benefit from multi-processing (depending upon the size of the parts).

Interesting that the output size was different -- I hadn't noticed that in my testing. It shouldn't matter with a difference that small, though.

[omegaman7]

@steveg32

Is MKV_INTERMEDIATE=1 set in your INI file?
I don't know about steve, but my INI does not contain such an entry. This is the very first time I've seen this error.

I even attempted to NOT keep the HD. Thereby compressing to 640Kbs. Still an error during mux.

I'm afraid I didn't keep the older version of BD rebuilder. I never do. My history with the program has been 95% perfect :)

Removing the DTS-HD XLL track via double click had no effect.

[01:43:49] BD Rebuilder v0.40.01 (beta)
- Source: STAR_TREK_S1D3
- Input BD size: 44.40 GB
- Approximate total content: [04:56:52.610]
- Target BD size: 23.10 GB
- Windows Version: 6.1 [7601]
- Quality: High Quality (Default), Two Pass
- Audio Settings: AC3=0 DTS=0 HD=1 Kbs=640
[01:43:51] PHASE ONE, Encoding
- [01:43:51] Processing: VID_00001 (1 of 64)
- [01:43:51] Extracting A/V streams [VID_00001]
- [01:44:01] Reencoding video [VID_00001]
- [01:44:01] Reencoding secondary video [TRK_02]
- Source Video: VC-1, 1920x1080
- Rate/Length: 23.976fps, 4,952 frames
- Bitrate: 1,171 Kbs
- [01:44:12] Reencoding: VID_00001, Pass 1 of 2
- [01:46:02] Reencoding: VID_00001, Pass 2 of 2
- [01:48:27] Video Encode complete
- [01:48:27] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- Track 4353 (eng): Keeping original audio
- [01:48:27] Multiplexing M2TS
- Error in attempt to multiplex: MUX_00001.meta
- Can't open file: R:\BD OUTPUT\WORKFILES\VID_00001_2.AVS.MKV
[01:48:30] - Failed to build structure, aborted

[jdobbs]

Yeah, I got it too when I tested a disc with a secondary. I'll fix it and post a correction version.

[colinhunt]

10-way encoding with MULTIPROCESS=1? You'd have to have at least 16 processors (as indicated by the NUMBER_OF_PROCESSORS environment variable) and 12GB of memory for that to happen. What kind of monster system are you using? I'm guessing it is the kind for which this feature was actually intended.
It's got two quad-core Xeons and 12 GB of RAM. That's a total of 8 physical cores, running 16 threads. So I'd say your guess is pretty much on the nose, if BD-RB thinks threads = processors.

So with MULTIPROCESS=3 you're seeing about a 50% increase in encoding speed, apparently.
The encoding speed during Pass 2 actually got faster when one of the three instances quit after finishing its part. That makes me very interested to see what happens during Pass 2 with 6-way split.

A single instance maxes out my CPU(s) but only during Pass 2. Pass 1 has been the biggest bottleneck in the past; a single instance barely breaks the 10% CPU usage barrier. No wonder 10 instances worked great for Pass 1 FPS-wise. Too bad the system ran out of RAM.

Obviously 10 processes was a bad idea -- can you try it with 6 or so?
10 might have been fine if the PC had had enough RAM. I'll set multiprocess=6 for the next test.

[MrT.]

failed doing a bd25 - movie and menu only, backup. It has a secondary video.

- Input BD size: 28.78 GB
- Approximate total content: [02:36:21.162]
- Target BD size: 22.95 GB
- Windows Version: 6.1 [7601]
- MOVIE and MENUS mode enabled
- Auto Quality: Good (Very Fast), ABR
- Audio Settings: AC3=1 DTS=1 HD=1 Kbs=640
[12:51:58] PHASE ONE, Encoding
- [12:51:58] Blanking: VID_00061 (1 of 74)
- [12:51:58] Blanking: VID_00062 (2 of 74)
- [12:51:58] Blanking: VID_00084 (3 of 74)
- [12:51:58] Blanking: VID_00085 (4 of 74)
- [12:51:58] Blanking: VID_50055 (5 of 74)
- [12:51:58] Blanking: VID_50056 (6 of 74)
- [12:51:58] Blanking: VID_50057 (7 of 74)
- [12:51:58] Blanking: VID_50058 (8 of 74)
- [12:51:58] Blanking: VID_50059 (9 of 74)
- [12:51:58] Blanking: VID_50060 (10 of 74)
- [12:51:58] Blanking: VID_50100 (11 of 74)
- [12:51:58] Blanking: VID_50101 (12 of 74)
- [12:51:58] Blanking: VID_50102 (13 of 74)
- [12:51:58] Blanking: VID_50103 (14 of 74)
- [12:51:58] Blanking: VID_50104 (15 of 74)
- [12:51:58] Blanking: VID_50105 (16 of 74)
- [12:51:58] Blanking: VID_50106 (17 of 74)
- [12:51:58] Blanking: VID_50126 (18 of 74)
- [12:51:59] Processing: VID_50201 (19 of 74)
- [12:51:59] Extracting A/V streams [VID_50201]
- [13:10:29] Reencoding video [VID_50201]
- [13:10:29] Reencoding secondary video [TRK_02]
- Source Video: VC-1, 1920x1080
- Rate/Length: 23.976fps, 148,191 frames
- Bitrate: 24,690 Kbs
- [13:22:03] Reencoding: VID_50201, Pass 1 of 1
- [14:17:33] Video Encode complete
- [14:17:33] Processing audio tracks
- Track 4352 (eng): Keeping original audio
- [14:17:33] Multiplexing M2TS
- Error in attempt to multiplex: MUX_50201.meta
- Can't open file: C:\OUTPUT\WORKFILES\VID_50201_2.AVS.MKV
[14:17:37] - Failed to build structure, aborted
-------------------------------
VERSION=v0.40.01 (beta)
SOURCE_SIZE=30897118762
SOURCE_VIDEO_SIZE=30397298688
TARGET_SIZE=24641536000
REDUCTION=.79420596460864
RESIZE_1080=0
AUDIO_TO_KEEP=eng;
KEEP_HD_AUDIO=-1
SUBS_TO_KEEP=eng;fra;fre;
BACKUP_MODE=0
MOVIEONLY_TYPE=0
USE_LAVF=0
MULTIPROCESS=1
QUICK=0
ENCODE_STEP=4
[50201]
AUDIO=100000
PGS=10101
VIDEO2=333898650.210084
V2MBRATE=7000
M2TS_TARGET=24141715926
RATE=24690

[omegaman7]

Secondary video eh? Apparently not an isolated problem. I wonder what causes it.

[jdobbs]

@MrT

That was already reported, I already noted that I was able to repeat it, I also said I'll be releasing an interim that fixes it (two posts up from yours).

[jdobbs]

I have updated the first post of this thread with links to a new version of BD-RB (v0.49.02). Changes for this release:- Corrected an issue introduced in v0.40.1 in
which an error occured during muxing of
streams containing secondary video.
- Made modifications to the algorithm used for
multi-processing instance count selection.
- Changed the label used in INF files that logs
the number of instances used during encoding
to prevent confusion with that in the INI.
- Other minor corrections and cosmetic fixes.

[omegaman7]

Thank you JD. Just in time for another test :D