Hardware considerations for BD Rebuilder

[RobertM]

So looks like it took 1:24 minutes compared to the original encode which was 1:23

So it took one minute longer??


Not so fast, Hombre ;) Careful jumping to conclusions. Let's break this down...


Target HDD dT SSD dT
Start 15:46:22 19:56:19
Extract 15:46:22 0:02:52 19:56:19 0:03:28
Pass1 15:49:14 0:25:07 19:59:47 0:25:05
Pass2 16:14:21 0:52:52 20:24:52 0:53:00
Audio 17:07:13 0:02:04 21:17:52 0:02:03
Rebuild 17:09:17 0:00:35 21:19:55 0:00:08
Cleanup 17:09:52 0:00:00 21:20:03 0:00:01
Finish 17:09:52 1:23:30 21:20:04 1:23:45


First, the difference is actually 15 sec, not 1 minute, when you don't round off the seconds.

Your extract time has increased, since you are extracting from a slower drive. This is not unexpected.

Your re-encode times are virtually unchanged. Further proof that your processor, fast as it is, is the bottleneck here.

Your rebuild time has dropped from 35 sec to 8 sec. This is a substantial improvement, since you are writing to the SSD array.

It looks like the reason why you don't see an overall gain is that your test is a bit unbalanced. You've got the large job (extraction of 25GB) on the slow drive and a small job (4.36GB rebuild) on the fast drive. So you've got all the deficits but few gains. Try repeating this test, but rebuild to a 25GB target size, and see what happens.



When I first start computer all fans go into high mode for about 10 seconds then they throttle down and when I start reincode they do throttle up again so I do believe the water cooler is working ok??
Yep, that sounds like it's functioning as it should.

[blusky]

Thanks RobertM for breaking that down . Makes sense . I will do a comparison later today to a BD25 size and see how it does.

Will let you know.

[blusky]

Hello RobertM ,
Here is the log file for the BD25 encode and I will include the configini file so there is no confusion on my settings:

----------------------
[04/29/13] BD Rebuilder v0.43.06 (beta)
[15:24:47] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[15:24:47] PHASE ONE, Encoding
- [15:24:47] Processing: VID_00800 (1 of 1)
- [15:24:47] Extracting A/V streams [VID_00800]
- [15:28:14] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [15:28:14] Reencoding: VID_00800, Pass 1 of 1
- [15:54:43] Video Encode complete
- [15:54:43] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[15:54:43]PHASE ONE complete
[15:54:43]PHASE TWO - Rebuild Started
- [15:54:43] Rebuilding BD file Structure
[15:55:11] - Encode and Rebuild complete
- WORKFILES folder removed.
[15:55:12] JOB: MEN_IN_BLACK_3 finished.



[Options]
VERSION=0.43.0.6
MODE=3
ENCODE_QUALITY=0
ONEPASS_ENCODING=2
AUTO_QUALITY=1
TARGET_SIZE=23500
AUDIO_TO_KEEP=eng;
SUBS_TO_KEEP=eng;
SD_CONVERT=0
OPEN_GOP=0
RESIZE_1080=0
DEINTERLACE=1
SD_TO_1080=0
CONVERT_WIDE=0
DTS_REENCODE=0
AC3_REENCODE=0
AC3_640=1
AC3_192=0
KEEP_HD_AUDIO=0
AVCHD=1
REMOVE_WORKFILES=1
MOVIE_ONLY_LOOP=1
REMOVE_OUTPUT=0
USE_FILTERS=0
BDMV_CERT_ONLY=0
USE_LAVF=0
IVTC_PULLDOWN=1
ASSUME_DVD_PAL=0
UNMASK_CHAPTER=1
COMPLETION_BEEP=0
DGDECNV=0
OUTPUT_3D=0
AUDIO_TRACK_LIMIT=1
SUBTITLE_TRACK_LIMIT=1
CUSTOM_TARGET_SIZE=23500
PRIORITY_CLASS=1
QUICK_EXTRAS=1
MOVIEONLY_TYPE=0
ALTCRF=20
ALT_TARGET=8096
ALTMETHOD=2
ALTAUTOCROP=0
[Paths]
SOURCE_PATH=H:\
WORKING_PATH=F:\WORKING\

I left the source folder and working folder in the same location as previous test. So it looks like it took 30:29 seconds to encode.

Next I will do the same encode but swap the Source file to the Raid 0 and the working file to the 4GB HDD and see what that does.

Thanks for your help.

[blusky]

Here is the same test but the source file is on the raid 0 and the working file is on the 4TB HDD.

----------------------
[04/29/13] BD Rebuilder v0.43.06 (beta)
[16:18:25] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[16:18:25] PHASE ONE, Encoding
- [16:18:25] Processing: VID_00800 (1 of 1)
- [16:18:25] Extracting A/V streams [VID_00800]
- [16:21:08] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [16:21:08] Reencoding: VID_00800, Pass 1 of 1
- [16:48:42] Video Encode complete
- [16:48:42] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[16:48:42]PHASE ONE complete
[16:48:42]PHASE TWO - Rebuild Started
- [16:48:42] Rebuilding BD file Structure
[16:51:34] - Encode and Rebuild complete
- WORKFILES folder removed.
[16:51:35] JOB: MEN_IN_BLACK_3 finished.

So looks like it took 33:10 seconds. So looks like the source file on the 4TB HDD and the working folder on the Raid 0 was faster 33:10 - 30:29=2:41. interesting.

For the Final test I will put the folders back to original since it was faster, Source folder on the 4TB HDD and the Working folder on the Raid 0 and I will add MULTIPROCESS=4
in the confi.ini folder and see if that works better.

Here is the final test. The Source folder on the 4TB HDD and the Working folder on the Raid 0 and I will add MULTIPROCESS=4

here is the log:

----------------------
[04/29/13] BD Rebuilder v0.43.06 (beta)
[17:14:30] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[17:14:30] PHASE ONE, Encoding
- [17:14:30] Processing: VID_00800 (1 of 1)
- [17:14:30] Extracting A/V streams [VID_00800]
- [17:18:04] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [17:18:04] Reencoding: VID_00800, Pass 1 of 1
- [17:45:42] Video Encode complete
- [17:45:42] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[17:45:42]PHASE ONE complete
[17:45:42]PHASE TWO - Rebuild Started
- [17:45:42] Rebuilding BD file Structure
[17:46:10] - Encode and Rebuild complete
- WORKFILES folder removed.
[17:46:11] JOB: MEN_IN_BLACK_3 finished.

This time it took 31:41

Can anyone please verify that the MULTIPROCCESS=4 is working?? Here is the Config.ini file::


[Options]
VERSION=0.43.0.6
MODE=3
ENCODE_QUALITY=0
ONEPASS_ENCODING=2
AUTO_QUALITY=1
TARGET_SIZE=23500
AUDIO_TO_KEEP=eng;
SUBS_TO_KEEP=eng;
SD_CONVERT=0
OPEN_GOP=0
RESIZE_1080=0
DEINTERLACE=1
SD_TO_1080=0
CONVERT_WIDE=0
DTS_REENCODE=0
AC3_REENCODE=0
AC3_640=1
AC3_192=0
KEEP_HD_AUDIO=0
AVCHD=1
REMOVE_WORKFILES=1
MOVIE_ONLY_LOOP=1
REMOVE_OUTPUT=0
USE_FILTERS=0
BDMV_CERT_ONLY=0
USE_LAVF=0
IVTC_PULLDOWN=1
ASSUME_DVD_PAL=0
UNMASK_CHAPTER=1
COMPLETION_BEEP=0
DGDECNV=0
OUTPUT_3D=0
AUDIO_TRACK_LIMIT=1
SUBTITLE_TRACK_LIMIT=1
CUSTOM_TARGET_SIZE=23500
PRIORITY_CLASS=1
QUICK_EXTRAS=1
MOVIEONLY_TYPE=0
ALTCRF=20
ALT_TARGET=8096
ALTMETHOD=2
ALTAUTOCROP=0
[Paths]
SOURCE_PATH=H:\
WORKING_PATH=F:\WORKING\
MULTIPROCESS=4

[gonca]

Hello Blusky
Sorry I took so long to respond but I just got home from work.
First, antivirus may be checking every new file, slowing it down.
Second, for clarity of comparison here are some of my specs
SOURCE DRIVE Vertex Plus SSD SATA 2
WORKING DRIVE Vertex Plus SSD SATA 2
CPU i7 3930K @4.4GHZ
COOLER H100
VIDEO GTX680 with DGDecNV for frame serving / decoding.
I use hwmonitor for temps, mine are at mid 60's Celsius on balanced, so yours are about right, and they look darn good.

I ran a encode withe DGDecNV and your settings for BD5 so you can judge your times.
Pass 1 cpu> 45 to 50%
Pass 2 cpu>100%

[04/29/13] BD Rebuilder v0.43.06 (beta)
[16:39:36] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 4.36 GB
- Windows Version: 6.1 [7601]
- MOVIE-ONLY mode enabled
- Auto Quality: High Quality (Default), Two Pass
- Decoding/Frame serving: DGDecNV [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[16:39:36] PHASE ONE, Encoding
- [16:39:36] Processing: VID_00800 (1 of 1)
- [16:39:36] Extracting A/V streams [VID_00800]
- [16:42:36] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 4,744 Kbs
- [16:42:36] Reencoding: VID_00800, Pass 1 of 2
- [17:02:56] Reencoding: VID_00800, Pass 2 of 2
- [17:42:35] Video Encode complete
- [17:42:35] Processing audio tracks
- Track 4352 (eng): Reencoding audio to AC3...
[17:44:20]PHASE ONE complete
[17:44:20]PHASE TWO - Rebuild Started
- [17:44:20] Rebuilding AVCHD file Structure
[17:44:42] - Encode and Rebuild complete
- WORKFILES folder removed.
[17:44:42] JOB: MEN_IN_BLACK_3 finished.

Time for encode 1:05:06

I am running an encode with Directshow right now for a closer analysis of your settings.

[blusky]

Hello Blusky
Sorry I took so long to respond but I just got home from work.
First, antivirus may be checking every new file, slowing it down.
Second, for clarity of comparison here are some of my specs
SOURCE DRIVE Vertex Plus SSD SATA 2
WORKING DRIVE Vertex Plus SSD SATA 2
CPU i7 3930K @4.4GHZ
COOLER H100
VIDEO GTX680 with DGDecNV for frame serving / decoding.
I use hwmonitor for temps, mine are at mid 60's Celsius on balanced, so yours are about right, and they look darn good.

I ran a encode withe DGDecNV and your settings for BD5 so you can judge your times.
Pass 1 cpu> 45 to 50%
Pass 2 cpu>100%

[04/29/13] BD Rebuilder v0.43.06 (beta)
[16:39:36] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 4.36 GB
- Windows Version: 6.1 [7601]
- MOVIE-ONLY mode enabled
- Auto Quality: High Quality (Default), Two Pass
- Decoding/Frame serving: DGDecNV [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[16:39:36] PHASE ONE, Encoding
- [16:39:36] Processing: VID_00800 (1 of 1)
- [16:39:36] Extracting A/V streams [VID_00800]
- [16:42:36] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 4,744 Kbs
- [16:42:36] Reencoding: VID_00800, Pass 1 of 2
- [17:02:56] Reencoding: VID_00800, Pass 2 of 2
- [17:42:35] Video Encode complete
- [17:42:35] Processing audio tracks
- Track 4352 (eng): Reencoding audio to AC3...
[17:44:20]PHASE ONE complete
[17:44:20]PHASE TWO - Rebuild Started
- [17:44:20] Rebuilding AVCHD file Structure
[17:44:42] - Encode and Rebuild complete
- WORKFILES folder removed.
[17:44:42] JOB: MEN_IN_BLACK_3 finished.

Time for encode 1:05:06

I am running an encode with Directshow right now for a closer analysis of your settings.

Wow, Gonca that is a really good time. Thanks, I would be interested on how it does with directshow.
Do you know the settings I have to change in Bd Rebuilder so I can get better quality encodes.

In post # 104 I did a reincode to a BD25 using the MULTIPROCESS=4 in the Config.ini file. Can you please take a look at post 104 and see if you can confirm that indeed MULTIPROCESS=4 was working??

Thanks Gonca.

[gonca]

Blusky, this is the directshow encode

[04/29/13] BD Rebuilder v0.43.06 (beta)
[17:56:19] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 4.36 GB
- Windows Version: 6.1 [7601]
- MOVIE-ONLY mode enabled
- Auto Quality: High Quality (Default), Two Pass
- Decoding/Frame serving: DirectShow [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[17:56:19] PHASE ONE, Encoding
- [17:56:19] Processing: VID_00800 (1 of 1)
- [17:56:19] Extracting A/V streams [VID_00800]
- [17:57:59] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 4,744 Kbs
- [17:57:59] Reencoding: VID_00800, Pass 1 of 2
- [18:20:52] Reencoding: VID_00800, Pass 2 of 2
- [19:06:01] Video Encode complete
- [19:06:01] Processing audio tracks
- Track 4352 (eng): Reencoding audio to AC3...
[19:07:47]PHASE ONE complete
[19:07:47]PHASE TWO - Rebuild Started
- [19:07:47] Rebuilding AVCHD file Structure
[19:08:09] - Encode and Rebuild complete
- WORKFILES folder removed.
[19:08:10] JOB: MEN_IN_BLACK_3 finished.


1:12 approximately

[Options]
VERSION=0.43.0.6
MODE=3
MULTIPROCESS=1
ENCODE_QUALITY=2
ONEPASS_ENCODING=0
AUTO_QUALITY=1
TARGET_SIZE=4469
AUDIO_TO_KEEP=eng;
SUBS_TO_KEEP=eng;
SD_CONVERT=0
OPEN_GOP=0
RESIZE_1080=0
DEINTERLACE=1
SD_TO_1080=0
CONVERT_WIDE=0
DTS_REENCODE=0
AC3_REENCODE=0
AC3_640=1
AC3_192=0
KEEP_HD_AUDIO=0
AVCHD=1
REMOVE_WORKFILES=1
MOVIE_ONLY_LOOP=1
REMOVE_OUTPUT=0
USE_FILTERS=0
BDMV_CERT_ONLY=0
USE_LAVF=0
IVTC_PULLDOWN=1
ASSUME_DVD_PAL=0
UNMASK_CHAPTER=1
COMPLETION_BEEP=0
DGDECNV=0
OUTPUT_3D=0
AUDIO_TRACK_LIMIT=1
SUBTITLE_TRACK_LIMIT=1
CUSTOM_TARGET_SIZE=23500
PRIORITY_CLASS=1
QUICK_EXTRAS=1
MOVIEONLY_TYPE=0
ALTCRF=20
ALT_TARGET=8096
ALTMETHOD=2
ALTAUTOCROP=0
[Paths]
SOURCE_PATH=I:\MEN_IN_BLACK_3\
WORKING_PATH=W:\HD\

DGIndexNV=C:\Program Files (x86)\dgdecnv2044\DGIndexNV.exe
DGDecNV=C:\Program Files (x86)\dgdecnv2044\DGDecodeNV.dll

[gonca]

According to post #104 you are not running multiprocess=4

Try this *.ini. It is basically the one you posted except for the MULTIPROCESS and the source and working paths. Change these two paths to suit your system and give it a try. Please remember that our two systems are similar, but not identical.

[Options]
VERSION=0.43.0.6
MODE=3
MULTIPROCESS=1
ENCODE_QUALITY=2
ONEPASS_ENCODING=0
AUTO_QUALITY=1
TARGET_SIZE=4469
AUDIO_TO_KEEP=eng;
SUBS_TO_KEEP=eng;
SD_CONVERT=0
OPEN_GOP=0
RESIZE_1080=0
DEINTERLACE=1
SD_TO_1080=0
CONVERT_WIDE=0
DTS_REENCODE=0
AC3_REENCODE=0
AC3_640=1
AC3_192=0
KEEP_HD_AUDIO=0
AVCHD=1
REMOVE_WORKFILES=1
MOVIE_ONLY_LOOP=1
REMOVE_OUTPUT=0
USE_FILTERS=0
BDMV_CERT_ONLY=0
USE_LAVF=0
IVTC_PULLDOWN=1
ASSUME_DVD_PAL=0
UNMASK_CHAPTER=1
COMPLETION_BEEP=0
DGDECNV=0
OUTPUT_3D=0
AUDIO_TRACK_LIMIT=1
SUBTITLE_TRACK_LIMIT=1
CUSTOM_TARGET_SIZE=23500
PRIORITY_CLASS=1
QUICK_EXTRAS=1
MOVIEONLY_TYPE=0
ALTCRF=20
ALT_TARGET=8096
ALTMETHOD=2
ALTAUTOCROP=0
[Paths]
SOURCE_PATH=I:\MEN_IN_BLACK_3\
WORKING_PATH=W:\HD\

[gonca]

Blusky
I just took a closer look at your ini file in post 104.
Move the MULTIPROCESS=4 line above the [Paths] line. Your outside of the options section and so that setting is being ignored.

[QBhd]

Also I strongly suggest you forgoe using your HDD during any step of the process. Your SSD RAID array is stupid fast and should be used as both the source and the working drive. And yes you had the Multiprocess option in the wrong place as gonca has said.

QB

[blusky]

Thanks Gonca and QBhd for the help. Ok , now I see where my error is in placing the MULTIPROCESS=4 in the Ini. Thanks will fix that. Thanks for running the encode . Now I have a benchmark to see a similar system like mine.
I will run it again with the MULTIPROCESS=4 in the Ini and then put both the Source and working folders on the Raid 0 and see how that does. Will ,post back. thanks.

[blusky]

This test the Source folder stayed the same on the 4TB HDD and the Working folder was on the Raid 0 SSD's. I did see this time in Task manager 4 instances off H264 running using approx. a total of 8.2GB of memory.

What I am thinking, since I have 64GB of memory, I could even try maybe MULTIPROCESS=8 and see how much memory that uses and adjust accordingly.
I am not sure if I can assume since MULTIPROCESS=4 used 8.2GB of memory then MULTIPROCESS=8 will use approx. 16.4GB??
here is the log file:

----------------------
[04/29/13] BD Rebuilder v0.43.06 (beta)
[20:28:34] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[20:28:34] PHASE ONE, Encoding
- [20:28:34] Processing: VID_00800 (1 of 1)
- [20:28:34] Extracting A/V streams [VID_00800]
- [20:32:06] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [20:32:06] Reencoding: VID_00800, Pass 1 of 1
- [21:02:18] Video Encode complete
- [21:02:18] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[21:02:18]PHASE ONE complete
[21:02:18]PHASE TWO - Rebuild Started
- [21:02:18] Rebuilding BD file Structure
[21:08:09] - Aborted per user request

I had forgotten to delete the reincode from previous run that's why it said aborted per user request.

[gonca]

blusky
Change your MULTIPROCESS to 1. This makes it automatic, however you should still see a four way split. If on your normal settings the cpu gets maxed out then increasing the number could actually slow it down. You asked about increasing the quality, try High 2 pass

[blusky]

blusky
Change your MULTIPROCESS to 1. This makes it automatic, however you should still see a four way split. If on your normal settings the cpu gets maxed out then increasing the number could actually slow it down. You asked about increasing the quality, try High 2 pass

Thanks Gonca for all your help. Before I reads your post I did a multiprocess=12 to see if the memory use is linear and it looks like it is. This time I used 16.6GB memory(26%) but the processor was at 100% utilization.

Here is the log:

----------------------
[04/29/13] BD Rebuilder v0.43.06 (beta)
[21:33:55] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow [12-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[21:33:55] PHASE ONE, Encoding
- [21:33:55] Processing: VID_00800 (1 of 1)
- [21:33:55] Extracting A/V streams [VID_00800]
- [21:34:34] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [21:34:34] Reencoding: VID_00800, Pass 1 of 1
- [22:05:04] Video Encode complete
- [22:05:04] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[22:05:04]PHASE ONE complete
[22:05:04]PHASE TWO - Rebuild Started
- [22:05:04] Rebuilding BD file Structure
[22:05:32] - Encode and Rebuild complete
- WORKFILES folder removed.
[22:05:34] JOB: MEN_IN_BLACK_3 finished.
----------------------


This one took 31:05


The last test for the night will be a MULTIPROCESS=1

I use clone drive to mount the .iso file onto a virtual drive H, so I am not sure how to use both Source and working file from the raid 0 ssd's??

[blusky]

her is the test with MULTIPROCESS=1

----------------------
[04/29/13] BD Rebuilder v0.43.06 (beta)
[22:17:00] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[22:17:00] PHASE ONE, Encoding
- [22:17:00] Processing: VID_00800 (1 of 1)
- [22:17:00] Extracting A/V streams [VID_00800]
- [22:17:42] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [22:17:42] Reencoding: VID_00800, Pass 1 of 1
- [22:48:11] Video Encode complete
- [22:48:11] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[22:48:11]PHASE ONE complete
[22:48:11]PHASE TWO - Rebuild Started
- [22:48:11] Rebuilding BD file Structure
[22:48:39] - Encode and Rebuild complete
- WORKFILES folder removed.
[22:48:40] JOB: MEN_IN_BLACK_3 finished.


This one took 31:48 to encode. Looks like the MULTIPROCESS=12 was a little faster at 31:05

[QBhd]

When mounting an image it is actually read from the drive the image is located on. Even though it is from a virtual drive the actual image file location is where the information is read from. So just make sure your image is located on the SSD RAID array, mount it and set your working folder also on the SSD RAID array. Basically all your work should be done on the SSD RAID. Rip to the RAID, mount the image from the RAID and output your encodes to the RAID. Your array has more than enough speed to read and write at the same time and still be many times faster than my 4 x Caviar Black RAID. If you want a simple test. Take any single large file on your SSD RAID (mkv, m2ts, etc etc) and copy it in the same location (it will read and write at the same time) Like this:

http://i.imgur.com/vY0WaTh.png

This will give a real world benchmark of your system's performance for working on the same drive

BTW, the above image is my 4x500GB Caviar Black RAID0 short stroked at 4 x 80.74GB or 300GB Drive size:

http://i.imgur.com/PzwpPfB.png

Not saying you should Short Stroke SSD's (I don't think it would be a boost and may actually be harmful)... it is information for others who may be reading this with HDD's in RAID0

QB

[RobertM]

Blusky,

Your tests confirm that your SSD array is significantly faster at both the extraction and the rebuild phases - but it really shines during the final, rebuilding stage. The best workflow depends on the relative sizes of the source material to the target material, but it looks to be the following:

BD25 Backup: HDD>SSD
BD5 Backup: SSD>HDD
BD9 Backup: probably HDD>SSD

There was discussion about using the SSD for both source and target, but I didn't notice that scenario in the log reports.

You just need to decide what type of backups you'll do most often and set up BDRebuilder for that case.


BTW,... remember one of your original posts:
Reason , I am way overdue on building a computer. Still using a 2004 Old Dell demensions XPS computer and it took 28 hours to encode.

28 hours to approx 30 min!! You've come a long way, Buddy :)

[blusky]

Blusky,

Your tests confirm that your SSD array is significantly faster at both the extraction and the rebuild phases - but it really shines during the final, rebuilding stage. The best workflow depends on the relative sizes of the source material to the target material, but it looks to be the following:

BD25 Backup: HDD>SSD
BD5 Backup: SSD>HDD
BD9 Backup: probably HDD>SSD

There was discussion about using the SSD for both source and target, but I didn't notice that scenario in the log reports.

You just need to decide what type of backups you'll do most often and set up BDRebuilder for that case.


BTW,... remember one of your original posts:


28 hours to approx 30 min!! You've come a long way, Buddy :)

Thanks RobertM for your help. Yes we have come a long way.
I am pretty happy with both the performance of my rig and how well BD Rebuilder works thanks to Jdobbs. With my older computer I did have some issues but that was because it was old and probably had a lot of old codecs in it that caused conflicts.
I will keep this rig as mainly a Rebuilder rig as to avoid that issue in the future.
Thanks goes to QBhd for the explanation on how the Virtual clone drive works. So later on today I will put both Source and working on the Raid 0 SSD's and see how that works.

Interesting enough the MULTIPROCESS=12 seemed to work the best. Just for grins it did work up to MULTIPROCESS=16 but the times began to slow down. Above that I did get this error: ffdshow vc-1 set incorrectly 0. I am sure it is just to many H264 processes running and the processor cannot handle it??
Does not really matter since the times were slower but I like to put in all the details in case it may help others in the future.

Will post results of both source and working folders on Raid 0 SSD's.
Next I need to find an explanation of all the option settings that there are so I can set up BD Rebuilder to give the best quality rein code.

Thanks

[QBhd]

At best an HDD can read OR write at 120 MB/s. When it comes to extracting/rebuilding this will be the biggest bottleneck if it is used in any step in your workflow (rip>extraction>encode>rebuild). Your SSD RAID is easily 4-5 times this speed if not more. Like I mentioned above just do some simple copy tests with a large file (m2ts, ts, mkv, etc etc). Copy to the HDD from RAID, copy to the RAID from HDD and copy to the RAID from the RAID... compare the results. I have zero doubt that the final test will be insanely fast.

QB

[RobertM]

I have zero doubt that the final test will be insanely fast.

Parts of it. But the encode time is still limited by the processor, so that will not likely change. I'm guessing that he'll shave a minute or so off the total; not a huge difference, but real time savings, to be sure.

The other thing Blusky will want to consider is where does he want the final result to sit. If he's going to burn the backup right away and clean off the working files then the SDD is fine for both source and target. But if he takes a while to get around to burning them then the SDD will prove to be too small. The extra step (moving the file from SDD to HDD), plus the time involved, may make some of the extreme speed gains moot.

[blusky]

Ok RobertM and QBhd,

Here is the Log for both the Source and Working folder on the Raid 0 SSD's.

----------------------
[04/30/13] BD Rebuilder v0.43.06 (beta)
[13:16:12] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow [12-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[13:16:12] PHASE ONE, Encoding
- [13:16:12] Processing: VID_00800 (1 of 1)
- [13:16:12] Extracting A/V streams [VID_00800]
- [13:17:40] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [13:17:40] Reencoding: VID_00800, Pass 1 of 1
- [13:49:45] Video Encode complete
- [13:49:45] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[13:49:45]PHASE ONE complete
[13:49:45]PHASE TWO - Rebuild Started
- [13:49:45] Rebuilding BD file Structure
[13:50:14] - Encode and Rebuild complete
- WORKFILES folder removed.
[13:50:15] JOB: MEN_IN_BLACK_3 finished.


This one took 34:03 but I set the Multiprocess=12. Can"t understand why it took longer?? The original multiprocess=12 with the Source on the 4TB HDD and the Working on the Raid 0SSD's had a time of 31:05??

I will try one more time with the same setup but Multiprocess=1 and see.
I will try this again and set multiprocess=1 and then we can have a good comparison.

[jdobbs]

Ok RobertM and QBhd,

Here is the Log for both the Source and Working folder on the Raid 0 SSD's.

----------------------
[04/30/13] BD Rebuilder v0.43.06 (beta)
[13:16:12] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow [12-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[13:16:12] PHASE ONE, Encoding
- [13:16:12] Processing: VID_00800 (1 of 1)
- [13:16:12] Extracting A/V streams [VID_00800]
- [13:17:40] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [13:17:40] Reencoding: VID_00800, Pass 1 of 1
- [13:49:45] Video Encode complete
- [13:49:45] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[13:49:45]PHASE ONE complete
[13:49:45]PHASE TWO - Rebuild Started
- [13:49:45] Rebuilding BD file Structure
[13:50:14] - Encode and Rebuild complete
- WORKFILES folder removed.
[13:50:15] JOB: MEN_IN_BLACK_3 finished.


This one took 34:03 but I set the Multiprocess=12. Can"t understand why it took longer?? The original multiprocess=12 with the Source on the 4TB HDD and the Working on the Raid 0SSD's had a time of 31:05??

I will try one more time with the same setup but Multiprocess=1 and see.
I will try this again and set multiprocess=1 and then we can have a good comparison.Lots of things impact it. With as much memory as you have, I'm wouldn't be surprised if you ran it two times in a row the second run would be faster -- because the entire disc could be kept in cache.

[blusky]

Lots of things impact it. With as much memory as you have, I'm wouldn't be surprised if you ran it two times in a row the second run would be faster -- because the entire disc could be kept in cache.

Thanks Jdobbs for the info. Yes, there a lot of variables like you said. Trying to figure out what MULTIPROCESS works best for my system. The memory use seemed to be linear so I could calculate approximate memory use at a certain number. The highest I was able to go was 16. After that I did get an error message but I am pretty sure it's the processor that can't handle it.

[jdobbs]

Thanks Jdobbs for the info. Yes, there a lot of variables like you said. Trying to figure out what MULTIPROCESS works best for my system. The memory use seemed to be linear so I could calculate approximate memory use at a certain number. The highest I was able to go was 16. After that I did get an error message but I am pretty sure it's the processor that can't handle it.Do you have a CUDA enabled nVidia video card (your first post mentioned the GTX680)? If so, you could also see speed improvements with DGDecNV. MULTIPROCESS also works faster with it (because the file splitting isn't necessary).

[blusky]

here is the last test. Multiprocess=1

Log
----------------------
[04/30/13] BD Rebuilder v0.43.06 (beta)
[14:18:09] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Auto Quality: Good (Very Fast), ABR
- Decoding/Frame serving: DirectShow [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[14:18:09] PHASE ONE, Encoding
- [14:18:09] Processing: VID_00800 (1 of 1)
- [14:18:09] Extracting A/V streams [VID_00800]
- [14:18:48] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [14:18:48] Reencoding: VID_00800, Pass 1 of 1
- [14:51:41] Video Encode complete
- [14:51:41] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[14:51:41]PHASE ONE complete
[14:51:41]PHASE TWO - Rebuild Started
- [14:51:41] Rebuilding BD file Structure
[14:52:09] - Encode and Rebuild complete
- WORKFILES folder removed.
[14:52:10] JOB: MEN_IN_BLACK_3 finished.

This one took 34:01 and the last one with Multiprocess=1 was
31:48 Interesting.....

[Ch3vr0n]

how can multiprocess work faster with DGDecNV if the filesplitting isn't necessairy? Isn't multiprocess exactly the thing that splits the stream by running multi-x264 instances.

[blusky]

Quick question... do you have the AX1200i and the H100i connected? As an owner of an H80i I have always been curious what the Corsair Link software shows in that situation

QB

ps... it's just QB (couldn't just use QB as a username so I came up with QBhd)

QBhd,
Yes I do have the AX-12300i power supply and the H100i water cooler. Yesterday I downloaded the latest Corsair Link Software and installed it. here is what it looks like:

http://i1286.photobucket.com/albums/a605/blusky1980/Screenshot10_zps4423d881.png (http://s1286.photobucket.com/user/blusky1980/media/Screenshot10_zps4423d881.png.html)


http://i1286.photobucket.com/albums/a605/blusky1980/Screenshot11_zpsc14af93a.png (http://s1286.photobucket.com/user/blusky1980/media/Screenshot11_zpsc14af93a.png.html)

Interesting Proggy. Tells you all temps and fan speeds. You can change the color of the Led on the pump to signify temp. I have it sait to white= cool Blue= warm and Red= hot. So it will change according to temp. Can control fan speeds and other good uses.

I was going to install the Asus Ai suit II but have heard a lot of of peeps have had issues with it so I will wait a little.

Hope this helps.

[blusky]

how can multiprocess work faster with DGDecNV if the filesplitting isn't necessairy? Isn't multiprocess exactly the thing that splits the stream by running multi-x264 instances.

Sorry ch3vr0n. Just saw your post after I posted above. I cannot answer that question but hope others that can will.

[blusky]

Do you have a CUDA enabled nVidia video card (your first post mentioned the GTX680)? If so, you could also see speed improvements with DGDecNV. MULTIPROCESS also works faster with it (because the file splitting isn't necessary).

Yes Jdobbs, I do have an Nvidea GEFORCE GTX 680 GPU.
I have found an old post of yours on how to install it, just waiting on the License.

Thanks

[QBhd]

blusky... thanks for sharing the images of the Corsair Link. If and when my PSU dies (although that may still be a long time away, LOL it is a Corsair HX750) I will certainly be getting an i version. I have my H80i colors set to go from blue>green>red depending on temps... pretty cool little feature :)

QB

[jdobbs]

how can multiprocess work faster with DGDecNV if the filesplitting isn't necessairy? Isn't multiprocess exactly the thing that splits the stream by running multi-x264 instances. To do it with DirectshowSource you can't simply seek by frame number -- because DirectshowSource isn't frame accurate. So BD-RB has to break up the file into smaller parts that are aligned on true breaks based upon what is listed in the EPMAP. With DGDecNV all I have to do is provide each X264 instance with a "--seek" command line option to point to the correct frame and each instance can read simultaneously from the original source.

Having written the code that does all this, I would have thought my saying so would have been enough.

[Ch3vr0n]

Oh i wasn't questioning you mate, just wondering how it works. So basically having DGDecNV & multiprocess enabled would be the fastest way of processing on "any" system?

[jdobbs]

Oh i wasn't questioning you mate, just wondering how it works. So basically having DGDecNV & multiprocess enabled would be the fastest way of processing on "any" system? Yes. The indexing associated with DGDecNV would theoretically increase time -- but I set it up so it happens concurrently with the audio extraction. That way it adds virtually no time (because they are both accessing the same source -- and the physical reads are cached).

[Ch3vr0n]

Sweet, thx for enlightening me.

[blusky]

Yes. The indexing associated with DGDecNV would theoretically increase time -- but I set it up so it happens concurrently with the audio extraction. That way it adds virtually no time (because they are both accessing the same source -- and the physical reads are cached).

Thanks Jdobbs for the explanation. Trying to get a better understanding of the process.
I did get the license for DGDecNV. Just want to make sure I use the correct one.

Should I use the 64 Bit version of DGIndexNV.EXE and DGDecodeNV.dll.??
I am using Windows 8 64bit.

[gonca]

Blusky
I think you should stick with the 32 bit version

[blusky]

Blusky
I think you should stick with the 32 bit version
Ok , great will use the 32 bit version. Thanks Gonca. Can't test it out till Friday, work got in the way.

[jdobbs]

Blusky
I think you should stick with the 32 bit versionDefinitely.

[blusky]

Definitely.

Thanks Jdobbs.

will use the 32bit version.

Appreciate your help.:thanks:

[blusky]

Here is another test using BD25, High Quality pass 2 , Multiprocess=1 and using Decoding/Frame serving: Dgdecnv2045. I do have an Invidea GEFORCE GTX 680 Vpu. Both the Source and Working folder on the Raid 0 array. Same movie Men in Black 3.
Here is the log:
----------------------
[05/03/13] BD Rebuilder v0.43.06 (beta)
[14:05:01] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Quality: High Quality (Default), Two Pass
- Decoding/Frame serving: DGDecNV [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[14:05:01] PHASE ONE, Encoding
- [14:05:01] Processing: VID_00800 (1 of 1)
- [14:05:01] Extracting A/V streams [VID_00800]
- [14:08:14] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [14:08:14] Reencoding: VID_00800, Pass 1 of 2
- [14:30:18] Reencoding: VID_00800, Pass 2 of 2
- [16:09:02] Video Encode complete
- [16:09:02] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[16:09:02]PHASE ONE complete
[16:09:02]PHASE TWO - Rebuild Started
- [16:09:02] Rebuilding BD file Structure
[16:09:31] - Encode and Rebuild complete
- WORKFILES folder removed.
[16:09:32] JOB: MEN_IN_BLACK_3 finished.


This what I have observed. Both in pass 1 and pass 2 the processor was utilized at 100%. I did see 4x H264 and each used about 25% of the processor utilization. On pass 1 the GPU was utilized about 90%. Pass 2 was about 6%.

My question is this. Since the processor is being utilized at 100% in both pass 1 and 2 am I getting any benefit by using Multiprocess=1??

Here is a screen shot of pass 2 using GPU-Z. Unfortunately I did not get a screen shot of pass 1 that was using more GPU utilization as I thought pass 2 would use more GPU utilization.

http://i1286.photobucket.com/albums/a605/blusky1980/Screenshot15_zpsd92e2793.png (http://s1286.photobucket.com/user/blusky1980/media/Screenshot15_zpsd92e2793.png.html)

Here are my BD Rebuilder settings:

http://i1286.photobucket.com/albums/a605/blusky1980/Screenshot13_zps44de1344.png (http://s1286.photobucket.com/user/blusky1980/media/Screenshot13_zps44de1344.png.html)

[jdobbs]

The goal of MULTIPROCESS is to get your processors to 100%, that's when you are using all horsepower available. After you've hit 100% there is no benefit in going higher. At some point there will also be diminishing returns.

Also, a quote from Neuron2:DGDecNV uses the PureVideo engine, not the GPU cores, for decoding.

[blusky]

The goal of MULTIPROCESS is to get your processors to 100%, that's when you are using all horsepower available. After you've hit 100% there is no benefit in going higher. At some point there will also be diminishing returns.

Ok, Thanks jdobbs, I will give it another try same settings except remove Multiprocess. I am curious about my CPU utilization to see if it differs.

Thanks.

[jdobbs]

Ok, Thanks jdobbs, I will give it another try same settings except remove Multiprocess. I am curious about my CPU utilization to see if it differs.

Thanks.Also, note my edit to my last post... the GPU isn't being used by DGDecNV.

Another note: It's common for Pass 2 to use more CPU than Pass 1. You want to get to a point at which you are using 100% on both passes.

[blusky]

Also, note my edit to my last post... the GPU isn't being used by DGDecNV.

Another note: It's common for Pass 2 to use more CPU than Pass 1. You want to get to a point at which you are using 100% on both passes.

Ok, Thanks Jdobbs. That is what I was looking for. I am running the second test now with the no Multiprocess and I ran a CPU usage of about 82% to 95% on the 1st pass. 100% on the 2nd pass.
Multiprocess=1 gave me 100% on both passes so There is the setting I was looking for. Only used about 4GB of memory in Multiprocess=1

This is the part that I was missing[QUOTE]
a quote from Neuron2:
Quote:
DGDecNV uses the PureVideo engine, not the GPU cores, for decoding.



I will post the no Multiprocess reincode for comparison. Its almost done.
By the way I did the test that QBhd had suggested by copying a large file from Raid 0 to raid 0, Raid 0 to HDD and HDD to raid 0. The raid 0 to raid 0 was by far the speed winner with a transfer rate of 500MB's. So what I will do once the encode is complete I will burn to disk and deleted contents of working file.

Thanks for helping me understand it all. I still have a lot to learn.

Thanks Jdobbs:cool:

[blusky]

Here is a test using BD25, High Quality pass 2 , Multiprocess=1 and using Decoding/Frame serving: Dgdecnv2045. Both the Source and Working folder on the Raid 0 array. Same movie Men in Black 3.
Here is the log:
[14:05:01] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Quality: High Quality (Default), Two Pass
- Decoding/Frame serving: DGDecNV [4-way]
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[14:05:01] PHASE ONE, Encoding
- [14:05:01] Processing: VID_00800 (1 of 1)
- [14:05:01] Extracting A/V streams [VID_00800] - [14:08:14] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs - [14:08:14] Reencoding: VID_00800, Pass 1 of 2
- [14:30:18] Reencoding: VID_00800, Pass 2 of 2 -
[16:09:02] Video Encode complete - [16:09:02] Processing audio tracks
- Track 4352 (eng): Keeping original audio [16:09:02]PHASE ONE complete [16:09:02]PHASE TWO
- Rebuild Started - [16:09:02] Rebuilding BD file Structure [16:09:31]
- Encode and Rebuild complete
- WORKFILES folder removed. [16:09:32] JOB: MEN_IN_BLACK_3 finished]

Here is the no multiprocess:
[05/03/13] BD Rebuilder v0.43.06 (beta)
[17:52:40] Source: MEN_IN_BLACK_3_00800
- Input BD size: 26.88 GB
- Approximate total content: [01:45:50.385]
- Target BD size: 22.95 GB
- Windows Version: 6.2 [9200]
- MOVIE-ONLY mode enabled
- Quality: High Quality (Default), Two Pass
- Decoding/Frame serving: DGDecNV
- Audio Settings: AC3=0 DTS=0 HD=0 Kbs=640
[17:52:40] PHASE ONE, Encoding
- [17:52:40] Processing: VID_00800 (1 of 1)
- [17:52:40] Extracting A/V streams [VID_00800]
- [17:55:53] Reencoding video [VID_00800]
- Source Video: MPEG-4 (AVC), 1920x1080
- Rate/Length: 23.976fps, 152,257 frames
- Bitrate: 27,186 Kbs
- [17:55:53] Reencoding: VID_00800, Pass 1 of 2
- [18:19:57] Reencoding: VID_00800, Pass 2 of 2
- [19:54:58] Video Encode complete
- [19:54:58] Processing audio tracks
- Track 4352 (eng): Keeping original audio
[19:54:58]PHASE ONE complete
[19:54:58]PHASE TWO - Rebuild Started
- [19:54:58] Rebuilding BD file Structure
[19:55:27] - Encode and Rebuild complete
- WORKFILES folder removed.
[19:55:28] JOB: MEN_IN_BLACK_3 finished.



Interesting enough the no Multiprocess total time was 2:02 48 and the Multiprocess=1 was 2:04 31.

The no multiprocess was faster by 1 min. 42 seconds.

I would have thought that the Multiprocess=1 would have been faster??

The Multiprocess=1 did max out the CPU at 100% in pass 1 and pass 2.

[jdobbs]

So would I. Lots of variables. Cache, maybe, but it doesn't look like it? I think the only way to truly be sure is to reboot your machine between the two encodes.

Interesting that pass 1 was faster in the MULTIPROCESS encode, but MULTIPROCESS was slower in pass 2. Hmmm...

[QBhd]

I have done many test with multi process encoding using MeGUI. There is usually a loss of time when doing 2 encodes at the same time during the second pass. The reason for this is that the 2nd pass almost always uses 100%. So if you do two processes at the same time there is greater chance of saturation and a loss of efficiency. The way I figured this out was encoding episodes of TV... many smaller runs. I would queue up all the encodes and then run two workers. All the 1st passes I would split between the two workers (thus using 100% CPU for first pass) but ALL the 2nd passes would run on the same worker one after the other (again using 100% CPU). This would shave a lot of time during the 1st passes. Going to 3 workers for the 1st passes did not work well since like the 2nd passes the CPU was already at 100% and times would suffer. I think your attempts at multiprocess greatly improve the times of the first pass, but kill the times of the second pass and as such the overall time is either the same or worse. From my experiments, two processes for the 1st pass and one process for the second pass is the best way to boost encoding times. Now the question is, how do you achieve this with just One source file (my testing involved multiple files). I think it is do-able, but I will leave that up to the developer to say if it is or isn't :)

QB

[blusky]

I have done many test with multi process encoding using MeGUI. There is usually a loss of time when doing 2 encodes at the same time during the second pass. The reason for this is that the 2nd pass almost always uses 100%. So if you do two processes at the same time there is greater chance of saturation and a loss of efficiency. The way I figured this out was encoding episodes of TV... many smaller runs. I would queue up all the encodes and then run two workers. All the 1st passes I would split between the two workers (thus using 100% CPU for first pass) but ALL the 2nd passes would run on the same worker one after the other (again using 100% CPU). This would shave a lot of time during the 1st passes. Going to 3 workers for the 1st passes did not work well since like the 2nd passes the CPU was already at 100% and times would suffer. I think your attempts at multiprocess greatly improve the times of the first pass, but kill the times of the second pass and as such the overall time is either the same or worse. From my experiments, two processes for the 1st pass and one process for the second pass is the best way to boost encoding times. Now the question is, how do you achieve this with just One source file (my testing involved multiple files). I think it is do-able, but I will leave that up to the developer to say if it is or isn't :)

QB

Thanks for the info QBhd,
I could not figure out why it was slower in Multiprocess=1 than in no multiprocess.

[blusky]

So would I. Lots of variables. Cache, maybe, but it doesn't look like it? I think the only way to truly be sure is to reboot your machine between the two encodes.

Interesting that pass 1 was faster in the MULTIPROCESS encode, but MULTIPROCESS was slower in pass 2. Hmmm...

Thanks jdobbs, I was thinking that myself that it may be the Cache. I will try again and reboot between the encodes and see what that does.

[jdobbs]

Thanks jdobbs, I was thinking that myself that it may be the Cache. I will try again and reboot between the encodes and see what that does.It could be that your SSD drives are so fast that you hit the point of diminishing returns very early. You could also try MULTIPROCESS=2 and MULTIPROCESS=3. But, frankly, you're already to a point where the overall differences are very small.