DVD Rebuilder 1.28.2 e CCE with Core i7 Intel &vista 64: low load for CPU

[pippocalo]

DVD Rebuilder is able to work with multiple core processing.
However i did found that with two core processor (windows XP 32 bit) both processor are engaged quite to 80%...but when you have Core i7 with 4 core and 8 thread only a part of CPU power is engaged.

It is not important the setting "idle" or "normal" of priority execution; the load for core i7 remain under 40%.

Someone knows how to override this limitation?

Someone can show the speed reached?

My core i7 920 (6 Gb of Ram) vista 64 with CCE 1.xx (more fast than CCE 2.7X) shows a speed about "10" without overclocking, and, I repeat a load for Cpu about 40%

Bye

[jdobbs]

It's just that way that CCE was developed. DVD-RB has the ability to run multiple instances of a program (as it does with HC and other encoders) to use up all the available processing power -- but CCE also doesn't allow multiple instances of itself to run.

I personally don't know of a way to overcome it.

[steptoe]

Try changing your encoder to HCEnc and let DVD-RB choose to run multiple instances

You should then find that virtually all your CPU power is now concentrating on encoding

I had that when I used CCE, but HCEnc will hapilly take over your system and use as much of your CPU as it can get


I have mine set so it doesn't run multiple instances as I still want to use my system for other tasks, but HCEnc hogs everything when set to multiple instances, like it should do

[pippocalo]

Thanks for the answer ...moderator.

I agree with you if CCE don not allow more than one instances, no trick is possible;
but with Win XP and an Intel core 2 I got an improvement putting "on" multiple processor option and the load is about 70% for each CPU.
Is the S.O. that divide the load between the two cores?

Regards

[pippocalo]

Try changing your encoder to HCEnc and let DVD-RB choose to run multiple instances

You should then find that virtually all your CPU power is now concentrating on encoding

I had that when I used CCE, but HCEnc will hapilly take over your system and use as much of your CPU as it can get


I have mine set so it doesn't run multiple instances as I still want to use my system for other tasks, but HCEnc hogs everything when set to multiple instances, like it should do

Thanks also to you for the answer.
But the quality of HCenc is similar to CCE?

Bye

[turbojet]

One process of each of these can run simultaneously: CCE SP2, CCE SP 2.7, 2.6, 2.5

I've read SP2 and SP 2.7 use the same encoding engine. Running 2 processes is ~25% speedup on a Q6600 and runs 100% CPU, i7 would see a greater speedup. Would it be possible for DVD-RB to call one segment as CCE SP2 the next segment as CCE SP 2.7 and run at the same time if paths for both are set?

It would be a nice improvement but if a new version of DVD-RB is released I mostly hope for an option to ivtc by segment as it can greatly improve 1000's of DVD's. Sorry for the OT.

[jdobbs]

I have to believe you don't understand how DVD-Rebuilder currently works.

You don't need to IVTC because it's already instrinsic in the way DVD-RB reencodes. If it is telecined, DVD Rebuilder encodes it at the true underlying framerate (ignoring the telecining flags) and then reinserts the pulldown flags during rebuild. You end up with a one-for-one encode that matches the original frame-by-frame. This is infinitely better than running any kind of IVTC algorithm. It also creates output that exactly matches the original.

By the way, this process is also the reason DVD-RB is the ONLY (reencoding) backup solution that works with hybrid sources.

Would it be possible for DVD-RB to call one segment as CCE SP2 the next segment as CCE SP 2.7 and run at the same time if paths for both are set? Yes, and I'd thought about that possibility in the past. But I just don't trust it -- and am always worried that each of the segments might not match at a consistent quality level. I decided to leave it with a single CCE instance. I'm really surprised that Custom Technology hasn't modified it (especially at the current cost) to support more threads.

[turbojet]

I'm also surprised custom technologies hasn't improved more then what they did with SP2. Maybe it's a marketing scheme to sell their dedicated hardware solutions. I compared CCE SP2 vs CCE SP 2.7 in CBR/1 pass VBR/2 pass VBR the vaf's and mpv's came out to identical sizes to the byte. The vaf's also look identical in vbr bitrate allocation window, SP2 can open/use 2.7's vaf and vice versa as well. In a hex editor there is some differences but they are the same few bits repeated so maybe it's a header with a version?

I haven't gotten around to testing 2.5 and 2.67 but iirc those both have fewer options then SP2/2.7 which probably means differences and I remember they both were to prone to undersize quite a bit in certain situations.

DVD-RB handles hybrid the best possible way indeed but when it comes to hard telecined it spreads the bitrate over 29.97 fps when the footage is actually 23.976 fps, many DVD's are like this unfortunately. For example the fox animation domination shows (american dad, king of the hill, family guy, simpsons) are hard telecined and there's 150-180 minutes on a DVD. Compressing these to dvd5 looks horrendous without ivtc, with ivtc they look about as good as the original.

Family Guy, American Dad, South Park are especially good test materials because they also emphasize the issues of different ivtc methods. Decomb v5 leaves a lot of combing around horizontal lines and in mouths, v4 does as well but not nearly as much. SmartDecimate either leaves a lot of combing or removes some lines completely depending on settings. TIVTC leaves some combing in the mouths but you really have to pay attention to spot it.

Most syfy shows and older tv shows are also hard telecined. As for movies, the majority of non-blockbuster movies I've seen are hard telecined and even some of the blockbuster ones are hard telecined like Stay Alive (http://www.imdb.com/title/tt0441796/). These are R1 DVD's, I've heard hard telecining is more prominent in asia.

As more of the less prominent studios jump into BD we'll probably experience hard telecine main features as well. I've already seen it in the extras on BD's of blockbuster movies.

While I understand the average user may not understand what ivtc is. It would save 'advanced users' hours of manual work if they were able to set a segment to ivtc in DVD-RB maybe by a per segment setting in the advanced editor. Very briefly it would be adding an avisynth function and pulldown.

Also there's a rarer case of hard telecine where 25 fps is put on NTSC DVD's. Usually 29.97 -> 25 fps with repal and dgpulldown 25 -> 29.97 does the trick. This is found in some european, australian, indian, chinese movies.

[jdobbs]

Hard telecining is garbage and is just extremely bad authoring. No matter how much I modify DVD Rebuilder I can't make the original authors any smarter. My own experience is that hard telecining is very rare. Most of the examples I've seen are caused by people who record directly from television at a high bitrate, write to DVD format, and then try to reduce with DVD-RB.

I can look at some type of solution -- but unfortunately I don't think I've ever seen anything that really works well consistently with hard telecined sources.

[SquallMX]

I can look at some type of solution -- but unfortunately I don't think I've ever seen anything that really works well consistently with hard telecined sources.

CCE SP2 3:2 detection (2 pass) works great for hard telecine and hybrid sources, unfortunately i don't think that could be implemented on DVD Rebuilder.

[turbojet]

Hard telecining is garbage and is just extremely bad authoring. No matter how much I modify DVD Rebuilder I can't make the original authors any smarter. My own experience is that hard telecining is very rare. Most of the examples I've seen are caused by people who record directly from television at a high bitrate, write to DVD format, and then try to reduce with DVD-RB.

I agree its bad authoring. However who knows what would happen to some DVD's if the dvd studio tried IVTCing when they receive a hard telecined source. If you are only working with blockbuster movies you'll mostly notice hard telecine in deleted scenes, these would also benefit a bit from IVTC but it doesn't have as much impact as main features.

I can look at some type of solution -- but unfortunately I don't think I've ever seen anything that really works well consistently with hard telecined sources.

TIVTC has worked correctly on 100-150 sources for me. Outside of the animation domination shows the only one it didn't handle flawlessly is The Wire HBO series. However tivtc handled both better then decomb.dll, IT.dll, SmartDecimate.dll.

CCE SP2 3:2 detection (2 pass) works great for hard telecine and hybrid sources, unfortunately i don't think that could be implemented on DVD Rebuilder.

I just tested this and got combing and the wrong frames removed on 3 sources, 2 hard telecined, 1 film flagged but honored pulldown flags in dgindex. I've tried changing pulldown, and rate conversion with no luck, am I doing something wrong?

An alternative is avisynth detection (http://avisynth.org/mediawiki/Interlace_detection) which megui and autogk uses. Unfortunately megui only detected 1 of 3 of the sources correctly as hard telecined. The other 2, including the one that's really film on the DVD were detected as hybrid/mostly film and suggested to deinterlace. I can trace the problem back to IsCombed from decomb.dll. When there is none to very little motion it detects the frame as not combed and tweaking the threshold didn't help much. So probably not a good option. It's very simple to detect the hard telecine with your eyes (2 of 5 frames combed) but unfortunately can't be detected easily with algorithms.

Considering this a switch to use TFM().TDecimate() in avisynth per segment would fulfill my need.

[SquallMX]

I just tested this and got combing and the wrong frames removed on 3 sources, 2 hard telecined, 1 film flagged but honored pulldown flags in dgindex. I've tried changing pulldown, and rate conversion with no luck, am I doing something wrong?

Yes, default CCE settings are crappy as hell, try using my template:
http://rapidshare.com/files/283621316/NTSC_29.97_DVD_Hybrid_Compilant.txt

Original hybrid Source (From BtVS Season 5 Disc 6 R4 Hard Telecined):
http://rapidshare.com/files/283624213/Buffy.Original.m2v
Stream Type: Elementary
Profile: main@main
Frame Size: 720x480
Display Size: [not specified]
Aspect Ratio: 16:9 [3]
Frame Rate: 29.970030 fps
Video Type: NTSC
Frame Type: Interlaced
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order: Top
Coded Number: 1619
Playback Number: 1619
Frame Repeats: 0
Field Repeats: 0
VOB ID:
Cell ID:
Bitrate: 1.768 Mbps
Bitrate (Avg): 5.975 Mbps
Bitrate (Max): 8.528 Mbps
Timestamp:
Elapsed: 0:00:54
Remain: FINISH
FPS: 29.91
Info:

My default settings ("Progressive Frame Pairing: 60", and "Repeat Field Tolerance: 2") will encode as 30i/30p even when the amount of 30i/30p motion content is minimal (In the Buffy.m2v file, all the frames with title cards are encoded interlaced even when 95% of the frame is 24p). This should be the recomended option for true hybrid sources if you have enough bitrate since is free of jerky playback (worst case scenario some 24p frames will be encode as 30i).

My default settings sample (Mostly Video encoded):
http://rapidshare.com/files/283625833/Buffy.m2v
Stream Type: Elementary
Profile: main@main
Frame Size: 720x480
Display Size: [not specified]
Aspect Ratio: 16:9 [3]
Frame Rate: 29.970030 fps
Video Type: Video 66.10%
Frame Type: Progressive
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order: Top
Coded Number: 1491
Playback Number: 1619
Frame Repeats: 0
Field Repeats: 256


Higher settings ("Progressive Frame Pairing: 66", and "Repeat Field Tolerance: 4")" should have more preference for 24p (In the Buffy.Alt.m2v file, most of the frames with title cards are encoded as 24p, except the ones with high 30i motion content). This should be the recomended option for hard telecined/hard telecine with interlaced credits sources if you have low/mid bitrate (worst case scenario some 30i credits will be encode as 24p).

Alt settings sample (Mostly Film encoded):
http://rapidshare.com/files/283627435/Buffy.Alt.m2v
Stream Type: Elementary
Profile: main@main
Frame Size: 720x480
Display Size: [not specified]
Aspect Ratio: 16:9 [3]
Frame Rate: 29.970030 fps
Video Type: Film 68.94%
Frame Type: Progressive
Coding Type: B
Colorimetry: BT.470-2 B,G*
Frame Structure: Frame
Field Order: Top
Coded Number: 1376
Playback Number: 1619
Frame Repeats: 0
Field Repeats: 486

The main problem with CCE 3:2 Pulldown is that sometimes 30i content with low motion will be encoded as 30p, this can be fixed using ReStream (just uncheck progresive flag).

Note: If the source video is Botton Field encoded, you should change "Offset Line" from "0" to "1".

[turbojet]

Both settings work fairly well on the buffy sample however they didn't work very well on the same 3 sources I tried earlier. They all ended up 53-68% interlaced NTSC all should result in 100% progressive.

TIVTC with TFM().TDecimate() handled all but the buffy sample great. With the buffy sample there was a few instances where it removed frames when it had to pick 1 of 5 good ones. TFM().TDecimate(hybrid=1) dealt with this situation by blending frames where needed, however I could only notice 1 blended frame playing back at 23.976. This handled the other 3 flawlessly without any blended frames.

To sum it up cce's auto detect is safe but not very effective, it also adds quite a bit of encoding time. TFM().TDecimate() is very effective but not always safe with hybrid content. TFM().TDecimate(hybrid=1) is very effective and safer with hybrid content. For auto detection cce sp2's would be good, however I would still opt for the manual setting to ivtc with probably TFM().TDecimate(hybrid=1) just to be a little safer. As I can usually tell if the stream is hard telecined, hybrid or interlaced in a couple of minutes judging by the d2v stats and a quick look frame by frame.

[SquallMX]

I agree, for film content encoded as hard-telecine, manual IVTC is the best choice, hybrid content is a little more tricky so the best choice varies between sources, CCE 3:2 detection should be useful for encoding DVD extras with low bit-rate where TFM().TDecimate(hybrid=1) blending is to noticeable.

To sum it up cce's auto detect is safe but not very effective, it also adds quite a bit of encoding time.

On a modern PC the additional pass takes no more than 30 minutes (15 if you're using a quad core :eek:).

BTW, can you post one of your samples please? I want to try some tweakings on the CCE settings :thanks: .

[turbojet]

Trailer here. (http://www.sendspace.com/file/dwe92n) it's not one of the test samples I had used but it shows the same ~60% film issue even though it should be 100% film.