Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > (Auto) Gordian Knot

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th January 2010, 18:17   #1  |  Link
scooby108
Registered User
 
Join Date: Dec 2009
Posts: 4
XVID crash w/ multi-core CPU and 2-pass encode

I am having problems encoding with AutoGK on a new computer. VirtualDubMod keeps crashing either right away or when it reaches 99%. I think I have figured out why this is happening with AutoGK.

My setup:
64bit Intel Xeon E5520 Quad-core WITH Hyperthreading ENABLED
Windows 7 Enterprise 64-bit
4GB RAM
Latest stable releases of AutoGK and Gordian Knot

I believe the problem is with the number of threads used to encode with XVID. AutoGK automatically detects the number of threads to use at each encoder pass. And each time it sets the number of threads equal to the number of virtual CPUs. Because I'm using a quad-core with HT enabled, this works out to be 8 virtual CPUs. VirtualDubMod crashes whenever threads is set to 8.

By opening the XVID encoder settings before/after each pass and changing "threads" to 4 I am able to get AutoGK to limp along far enough to have it build its AviSynth script. Once I get the script I can set threads back to 4 and do a manual encode using that script. Single-pass encodings work fine if I can set the number of threads to 4.

Can there be a cap to the number of threads or a registry setting to limit # of threads? I read that you shouldn't need any more than 3-4 threads when encoding with XVID anyway because after that the gain is marginal. Is that true?
scooby108 is offline   Reply With Quote
Old 5th January 2010, 00:39   #2  |  Link
nurbs
Registered User
 
Join Date: Dec 2005
Posts: 1,460
Quote:
Originally Posted by scooby108 View Post
I read that you shouldn't need any more than 3-4 threads when encoding with XVID anyway because after that the gain is marginal. Is that true?
Don't know how you can limit the number, but this is true. Xvids threading is pretty inefficient.
nurbs is offline   Reply With Quote
Old 5th January 2010, 01:30   #3  |  Link
netmask
Registered User
 
netmask's Avatar
 
Join Date: Jan 2005
Location: Sydney Australia
Posts: 260
Quote:
My setup:
64bit Intel Xeon E5520 Quad-core WITH Hyperthreading ENABLED
Windows 7 Enterprise 64-bit
4GB RAM
Latest stable releases of AutoGK and Gordian Knot
Every box running Windows 7 64 bit I've had any experience with has had a minimum of 6GB RAM. Maybe it's a memory problem?
__________________
SONY 75" Full array 200Hz LED TV
Yamaha A1070 amp
Zidoo UHD3000, Popcorn Hour A-500
BeyonWiz PVR T3 & V2 (Enigma2 clone)
Chromecast
Pioneer Bluray BDP-150-K
Windows 7 Ultimate
QNAP NAS TS851
netmask is offline   Reply With Quote
Old 5th January 2010, 01:49   #4  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by netmask View Post
Every box running Windows 7 64 bit I've had any experience with has had a minimum of 6GB RAM. Maybe it's a memory problem?
Unless he is using 64-Bit Xvid (which would also require 64-VirtualDub), he is using 32-Bit Xvid. So there will be a 2 GB per-process memory limit anyway!

No matter how much RAM you have or how much memory your OS can address, the per-process limit for 32-Bit processes remains
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 5th January 2010 at 01:52.
LoRd_MuldeR is offline   Reply With Quote
Old 5th January 2010, 02:06   #5  |  Link
netmask
Registered User
 
netmask's Avatar
 
Join Date: Jan 2005
Location: Sydney Australia
Posts: 260
Quote:
Originally Posted by LoRd_MuldeR View Post
Unless he is using 64-Bit Xvid (which would also require 64-VirtualDub), he is using 32-Bit Xvid. So there will be a 2 GB per-process memory limit anyway!

No matter how much RAM you have or how much memory your OS can address, the per-process limit for 32-Bit processes remains
But the OP says he using 64 bit.
__________________
SONY 75" Full array 200Hz LED TV
Yamaha A1070 amp
Zidoo UHD3000, Popcorn Hour A-500
BeyonWiz PVR T3 & V2 (Enigma2 clone)
Chromecast
Pioneer Bluray BDP-150-K
Windows 7 Ultimate
QNAP NAS TS851
netmask is offline   Reply With Quote
Old 5th January 2010, 02:24   #6  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by netmask View Post
But the OP says he using 64 bit.
He says he's using 64-Bit Windows. He doesn't say he's using 64-Bit Xvid. That's a fundamental difference

But he mentions AutoGK/VirtualDubMod, which, as far as I know, doesn't have a 64-Bit version available. So I must assume he's using 32-Bit Xvid.

And as said before: Even on 64-Bit Windows with 4 GB of RAM or more, each 32-Bit process is limited to 2 GB of memory

My advice for his problem would be: Instead of using the discontinued/outdated VirtualDubMod he should update to an up-to-date VirtualDub!
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 5th January 2010 at 02:30.
LoRd_MuldeR is offline   Reply With Quote
Old 5th January 2010, 09:23   #7  |  Link
scooby108
Registered User
 
Join Date: Dec 2009
Posts: 4
I don't think VirtualDub can help this time

Quote:
Originally Posted by LoRd_MuldeR View Post
He says he's using 64-Bit Windows. He doesn't say he's using 64-Bit Xvid.
True indeed.

Quote:
Originally Posted by LoRd_MuldeR View Post
My advice for his problem would be: Instead of using the discontinued/outdated VirtualDubMod he should update to an up-to-date VirtualDub!
I have the latest stable VirtualDub (1.9.8) that I use for AVI processing, however, I cannot use it this case since AutoGK is converting from MPEG-2 sources. I must use VirtualDubMod for its MPEG-2 capabilities.

And I can use the scripts created by AutoGK (.avs and .vcf) to process, encode, and mux, as long as I set the threads back to 4 before I start. Makes having a queue useless since I have to babysit it and manually load the scripts into VDM to get the encoding done.
scooby108 is offline   Reply With Quote
Old 5th January 2010, 12:11   #8  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
You know that up-to-date VirtualDub supports Input-Plugins and that there's a MPEG-2 Plugin a available ???

No need to use an old and outdated fork of VirtualDub for MPEG-2 input

Also, as an alternative to VirtualDub(Mod) and VFW Codecs, you may want to give Avidemux a try...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 5th January 2010 at 13:22.
LoRd_MuldeR is offline   Reply With Quote
Old 5th January 2010, 19:52   #9  |  Link
scooby108
Registered User
 
Join Date: Dec 2009
Posts: 4
AutoGK and up-to-date VirtualDub with MPEG-2 plugin fails

Quote:
Originally Posted by LoRd_MuldeR View Post
You know that up-to-date VirtualDub supports Input-Plugins and that there's a MPEG-2 Plugin a available ???
I had no idea that there was a plugin for MPEG-2! Awesomeness!
However it still doesn't help me with AutoGK. I have used many tools for transcoding, but none as quick and easy as AutoGK. It takes 5-10 seconds to add a video to the queue in AutoGK whereas Avidemux takes 1-2 minutes. I really value the speed and simplicity.

Anyway, I tried to plug an up-to-date VirtualDub into my AutoGK install and get an error:


I looked at the switches for the GUI and command line versions of VirtualDub and neither of them have the "nowrite" switch.

Quote:
Originally Posted by LoRd_MuldeR View Post
Also, as an alternative to VirtualDub(Mod) and VFW Codecs, you may want to give Avidemux a try...
I'm not looking for a replacement for AutoGK, just an improvement. If I have to do things the hard (pronounced "slightly more time-intensive and not automatic") way, then I will. I was only hoping for an easy solution. I really like Avidemux, BTW.

Thanks for the help, MuldeR.
Attached Images
 
scooby108 is offline   Reply With Quote
Old 5th January 2010, 20:28   #10  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
As far as I know, AutoGK builds on top of Gordian Knot. And Gordian Knot uses VirtualDubMod. I don't think it can use up-to-date VirtualDub.

So if VirtualDubMod fails, your only option will be droppoing AutoGK/VDubMod and move the a less antiquated tool chain.

But most important: Did you try to encode manually from an up-to-date VirtualDub? Does it still crash or is the problem fixed now ???
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊
LoRd_MuldeR is offline   Reply With Quote
Old 5th January 2010, 20:53   #11  |  Link
scooby108
Registered User
 
Join Date: Dec 2009
Posts: 4
Guess I'll switch to Avidemux

Quote:
Originally Posted by LoRd_MuldeR View Post
But most important: Did you try to encode manually from an up-to-date VirtualDub? Does it still crash or is the problem fixed now ???
I direct you to my other comment:

Quote:
Originally Posted by scooby108 View Post
And I can use the scripts created by AutoGK (.avs and .vcf) to process, encode, and mux, as long as I set the threads back to 4 before I start. Makes having a queue useless since I have to babysit it and manually load the scripts into VDM to get the encoding done.
By VDM I meant VirtualDubMod. So, yes. The scripts work fine, VirtualDubMod works fine, VirtualDub works fine, XVID works fine, all fine EXCEPT when using 8 threads to encode.

Maybe Len0x will run across this post and put a option for the number of threads into AutoGK. I believe AutoMKV has the option and stores it with each item in the queue. And Avidemux has the multithreading option at least when using XVID or x264 (Disable, Auto, Custom). Something like that in AutoGK would be great.

I guess I'll have to go with Avidemux. That's what I use most of the time on Linux anyway.

Thanks again, MuldeR.
scooby108 is offline   Reply With Quote
Old 9th January 2010, 07:07   #12  |  Link
88keyz
Registered User
 
Join Date: Jun 2005
Location: Canada
Posts: 29
Seems like a stupid suggestion but switching to a single threaded version of Xvid will help. I was having the same problem with AutoGK after upgrading to an i7 processor and I solved it by going back to an older version of Xvid that didn't support multi-threaded encoding. Strangely thanks to Intel Turbo Tech the single threaded encoding isn't that much slower on my i7-860 than the multi-threaded Xvid. However it doesn't crash VirtualDubMod and your encodes will actually finish, so I guess in that sense its actually faster.

Xvid_1.1.3-27042008

On a side note, if anyone has Koepi's 1.1.3 VAQ build dated 27 Apr 2008 (filename Xvid-1.1.3-VAQ-27042008.exe) and would care to post it for download I would be grateful. I looked high and low for it on the web and all download links point back to Koepi's site and he has removed it for download.

Last edited by 88keyz; 9th January 2010 at 15:36. Reason: Add link to Xvid_1.1.3-27042008
88keyz is offline   Reply With Quote
Old 17th January 2010, 20:53   #13  |  Link
88keyz
Registered User
 
Join Date: Jun 2005
Location: Canada
Posts: 29
A quick follow-up. I found that Jawor's 1.3.-127 build works just fine with up to 8 threads in AutoGK. Not sure what the difference is but since I switched to this build I have not had a crash in AutoGK.

Get Jawor's Xvid builds here: Jawor's Xvid Binaries

or here: Jawor's Xvid 1.3.-127
88keyz is offline   Reply With Quote
Old 8th August 2010, 20:10   #14  |  Link
szabi
Registered User
 
Join Date: Nov 2004
Posts: 251
Quote:
Originally Posted by 88keyz View Post
Seems like a stupid suggestion but switching to a single threaded version of Xvid will help. I was having the same problem with AutoGK after upgrading to an i7 processor and I solved it by going back to an older version of Xvid that didn't support multi-threaded encoding. Strangely thanks to Intel Turbo Tech the single threaded encoding isn't that much slower on my i7-860 than the multi-threaded Xvid. However it doesn't crash VirtualDubMod and your encodes will actually finish, so I guess in that sense its actually faster.
I hope it will help me, because today I got a problem during encoding vdub and XviD 1.2.2.
I tried to use newer vdubmod, also the newest vdub (v1.9.9) but I got crash in both case.
I will try this thread control tomorrow.
Thnx for the idea.

EDIT: Ok it worked fine.

Bye
szabi

Last edited by szabi; 12th August 2010 at 09:10.
szabi is offline   Reply With Quote
Reply

Tags
crash, threads, virtualdubmod, xvid

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.