Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Programming and Hacking > Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 31st May 2011, 21:48   #281  |  Link
Raen
Registered User
 
Join Date: Apr 2011
Posts: 9
Thanks for the fast reply and also for the new CUE import wizard, seems to be working flawlessly so far Great work!

Quote:
Originally Posted by LoRd_MuldeR View Post
If other applications don't read the file correctly, then I cannot do much.

IMHO these applications should check for the UTF-8 BOM and treat the file like UTF-8, if the BOM is present.

But if changing the file extension helps, I might do.

However people that only know M3U files will surely be confused about M3U8 files
Well, when I encountered this situation and tried to work around it, I saw that Winamp had an option to save "M3U8 (Unicode)" playlists and figured out the difference between the two extensions.

Maybe if you change this text to "(.m3u8 - Unicode)" maybe people don't get confused about why LameXP is outputting some strange ".m3u8" playlists?


EDIT: I was playing some playlists that were created with LameXP v3.18 a while ago and noticed that these playlists, although being .m3u, recognize files with special characters.
So, there must be some difference between the v3.18's .m3u creation method and the v4.xx's.

Last edited by Raen; 1st June 2011 at 19:55.
Raen is offline   Reply With Quote
Old 2nd June 2011, 01:53   #282  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by Raen View Post
EDIT: I was playing some playlists that were created with LameXP v3.18 a while ago and noticed that these playlists, although being .m3u, recognize files with special characters.
So, there must be some difference between the v3.18's .m3u creation method and the v4.xx's.
LameXP 3.xx did not support Unicode at all


All strings were simply in the local 8-Bit encoding, i.e. whatever Codepage happened to be configured for Non-Unicode applications on the individual computer.

If you have luck, the 'special character' can be represent in your local 8-Bit encoding, so LameXP 3.xx will be able to display it correctly and save it to the M3U file (again in a local 8-Bit encoding).

And, if you have even more luck, then the other application (e.g. Winamp) will interpret the M3U file with the same local 8-Bit encoding as LameXP and thus will read the 'special character' correctly.

But there is absolutely no guarantee it will work as desired! Also with a local 8-Bit encoding, it is never possible to have characters from different Codepages at the same time...


LameXP v4.xx provides proper Unicode support. Internally everything is represented as Unicode/UTF-16. Only question is: How do we export strings to other programs?

I have now changed the playlist creation in the following way:

* If all file names in the playlist can be represented correctly as Latin-1 (ISO 8859-1) then the M3U file is written as plain Latin-1 text with a '.m3u' extension.

* If there is at least one file name that contains Unicode (Non-Latin1) characters, then the M3U file is written as UTF8-encoded text starting with a BOM and with a '.m3u8' extension.


So can you please try again with Build #560, which is now available via auto-update?
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 2nd June 2011 at 02:09.
LoRd_MuldeR is offline   Reply With Quote
Old 2nd June 2011, 16:14   #283  |  Link
Raen
Registered User
 
Join Date: Apr 2011
Posts: 9
Yes, looks like I got some luck with those older playlists, because I found another ones that targeted files that had non-latin characters (cyrillic, japanese, etc) and those didn't play, only the ones that had latin special characters (accents, etc) played.

Quote:
Originally Posted by LoRd_MuldeR View Post
So can you please try again with Build #560, which is now available via auto-update?
Can you/Will you put the new build also in the Downloads page?
If I can avoid installing, the better, I prefer the portable/zip package

Last edited by Raen; 13th June 2011 at 23:20.
Raen is offline   Reply With Quote
Old 2nd June 2011, 19:59   #284  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by Raen View Post
Can you/Will you put the new build also in the Downloads page?
If I can avoid installing, the better, I prefer the portable/zip package
Pre-release builds of LameXP are available from the following location:
http://sourceforge.net/projects/lame...pshots (BETA)/

However in order to get the latest one, you should always run auto-update. I am too lazy to upload every build twice, so the auto-update server is updated more frequently
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 2nd June 2011 at 21:47.
LoRd_MuldeR is offline   Reply With Quote
Old 3rd June 2011, 22:26   #285  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Today I made a LameXP build with the recent technology preview of Qt v4.8.0:
http://www.mediafire.com/file/xeg7st....Build-561.zip

You are welcome to report any regressions or improvements that you may encounter
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 3rd June 2011 at 22:37.
LoRd_MuldeR is offline   Reply With Quote
Old 4th June 2011, 21:40   #286  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
LameXP v4.02 Beta-7:

Implemented a custom QFileIconProvider, which (hopefully) makes the "Output Folder" view a bit faster.
Note: This build, in contrast to the one from my previous post, has been made with Qt 4.7.3 again!
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 5th June 2011 at 20:32.
LoRd_MuldeR is offline   Reply With Quote
Old 5th June 2011, 09:28   #287  |  Link
codres
Registered User
 
Join Date: Dec 2002
Location: Galati
Posts: 11
I'm getting this error when I try to encode a 5 channel ac3 file to joint stereo. Seems that the resulted wav is aroung 5Gb. Maybe because the huge size ?

Quote:
LameXP v4.01 (Build #418), compiled at 2011-04-04

-------------------------------

C:/DOCUME~1/ADMINI~1/LOCALS~1/Temp/9d3c8ffa6eec43b295a520c4348c6d49/tool_valdec.exe "H:\Bietul Ioanide.1979 T01 3_2ch 448Kbps DELAY 0ms.ac3" -w F:\\c8e18d6f24704bd6bca7dae03f4a1902.wav

Opening audio output PCM16 3/2.1 (5.1) 48000...
---------------------------------------
Frames/errors: 264865/0
System time: 128594ms
Process time: 126578ms
Approx. 1.49% realtime CPU usage

Exited with code: 0x0000

-------------------------------

C:/DOCUME~1/ADMINI~1/LOCALS~1/Temp/9d3c8ffa6eec43b295a520c4348c6d49/tool_sox.exe -V3 -S --guard --temp . F:\\c8e18d6f24704bd6bca7dae03f4a1902.wav -c2 F:\\78d55f307e5549c7b3e0f551cafc1926.wav

C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\9d3c8ffa6eec43b295a520c4348c6d49\tool_sox.exe: SoX v14.3.2
C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\9d3c8ffa6eec43b295a520c4348c6d49\tool_sox.exe FAIL formats: can't open input file `F:\\c8e18d6f24704bd6bca7dae03f4a1902.wav': WAVE: RIFF header not found

Exited with code: 0x0002
__________________
Codres A Bogdan
codres is offline   Reply With Quote
Old 5th June 2011, 13:01   #288  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Wave/RIFF files cannot exceed a size of 4 GB. That's a limitation of the RIFF file format, because the size field is 32-Bit

As far as I know, ValibDec will create a RF64 file, if the size exceeds 4 GB. Unfortunately most tools, including SoX, do NOT support RF64 files yet...

(Indeed there is no RIFF header in this case, only a RF64 header)
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 5th June 2011 at 19:52.
LoRd_MuldeR is offline   Reply With Quote
Old 5th June 2011, 13:41   #289  |  Link
codres
Registered User
 
Join Date: Dec 2002
Location: Galati
Posts: 11
Ok, thanks for clearing some things. I manage to reencode the file with eac3to.
__________________
Codres A Bogdan
codres is offline   Reply With Quote
Old 5th June 2011, 20:35   #290  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
LameXP v4.02 RC-1:

Last chance to report bugs that you don't want to see in the next release version
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 14th June 2011 at 18:52.
LoRd_MuldeR is offline   Reply With Quote
Old 11th June 2011, 01:56   #291  |  Link
Chimel
Registered User
 
Chimel's Avatar
 
Join Date: Apr 2010
Posts: 3
Simpler Add UI

Not a bug, but a suggestion to simplify the Add File(s)/Folder features:
Why not combine them into a single Add Files button and let the user browse and select either files or a folder?
Maybe with a "Add recursively" checkbox in the Browse dialog box when a folder is selected.

Personally I use only the "File|Open Folder Recursively" menu, it's been a bit disappointing to see the Add Folder button disappear. That's why it would be great to be able to select either files or folders from the same button in the main UI, I don't really understand why there needs to be 3 different menu items to add files. And if it's a folder, the recursive checkbox should be selected by default. Very low priority, I just thought it would improve usability and be easier to maintain just one Add feature too.

There's one little bug that bothers me, although I'm sure it's meant to be by design: When adding files, for instance a large recursive folder, each of the file names is displayed on the forefront in the middle of the screen, so you can't switch to another app like a browser or video player while it's adding the files. Which in my case took 3 minutes for over 400 files (I had another CPU-intensive app running at the same time.)
Is it possible to display the file names only if the app in focus is LameXP?

FYI, this is how I use LameXP: Usually once a week I rip my new CDs with WMP into lossless WMAs, takes about 2 minutes per CD max. Then I move all the CD folders into a single folder outside the WMP library and run LameXP on this folder recursively. Then the MP3s are moved to a separate folder where I can pick them up for mobile usage, the WMAs are moved back to the WMP library. I use robocopy /mov and /move to keep the same structure. If I could specify the folder to add as a parameter, everything could be scripted.
Chimel is offline   Reply With Quote
Old 11th June 2011, 02:48   #292  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by Chimel View Post
Not a bug, but a suggestion to simplify the Add File(s)/Folder features:
Why not combine them into a single Add Files button and let the user browse and select either files or a folder?
Maybe with a "Add recursively" checkbox in the Browse dialog box when a folder is selected.

Personally I use only the "File|Open Folder Recursively" menu, it's been a bit disappointing to see the Add Folder button disappear. That's why it would be great to be able to select either files or folders from the same button in the main UI, I don't really understand why there needs to be 3 different menu items to add files. And if it's a folder, the recursive checkbox should be selected by default. Very low priority, I just thought it would improve usability and be easier to maintain just one Add feature too.
A combined "add files/folder" feature would be hard to implement, because there are distinct "open" dialog boxes for selecting files and folders.

However you may add folders via drag&drop. The "drop box" may also be helpful here...

Quote:
Originally Posted by Chimel View Post
There's one little bug that bothers me, although I'm sure it's meant to be by design: When adding files, for instance a large recursive folder, each of the file names is displayed on the forefront in the middle of the screen, so you can't switch to another app like a browser or video player while it's adding the files. Which in my case took 3 minutes for over 400 files (I had another CPU-intensive app running at the same time.)
Is it possible to display the file names only if the app in focus is LameXP?
Not currently.

Quote:
Originally Posted by Chimel View Post
FYI, this is how I use LameXP: Usually once a week I rip my new CDs with WMP into lossless WMAs, takes about 2 minutes per CD max. Then I move all the CD folders into a single folder outside the WMP library and run LameXP on this folder recursively. Then the MP3s are moved to a separate folder where I can pick them up for mobile usage, the WMAs are moved back to the WMP library. I use robocopy /mov and /move to keep the same structure. If I could specify the folder to add as a parameter, everything could be scripted.
Currently individual files can be added via command-line using "--add <filename>".

Shouldn't be too hard to add "--add-folder <folder>" and "--add-recursive <folder>" options in one of the next versions.
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊
LoRd_MuldeR is offline   Reply With Quote
Old 11th June 2011, 19:04   #293  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
LameXP v4.02 RC-3:

Added "--add-folder <path>" and "--add-recursive <path>" command-line switches.
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 14th June 2011 at 18:52.
LoRd_MuldeR is offline   Reply With Quote
Old 13th June 2011, 08:04   #294  |  Link
phideaux3
Registered User
 
Join Date: Dec 2002
Location: NYC
Posts: 3
First, just wanted to say how much I appreciate your having created this tool and continuing to maintain it. I've been using it for a few years and think it's got to be the best for my needs.

In an earlier comment, you said “try again with Build #560, which is now available via auto-update” – not sure what you meant by this. Should I be able to run ‘Check for Updates’ from the app and it should grab the new build? If so, it didn’t work. It kept saying 418 was the latest.

Anyway, the reason I came to this forum was because 418 kept crashing my system (W7Usp1 64-bit). I was encoding to mp3 it on a quad-core with a queue of over 2k files. I reduced the threads to 3, and it didn’t crash, but over 50% of the encodes failed with error code 3 or 5 and a very small or zero-byte mp3 file.

I installed Build 574 and reduced the queue to fewer than 600 and left threads at 3. The app died with “Unhandled exception error, application will exit.” after 313 files completed.

I ran another queue of about 250 files, all finished except two, which failed with

PROCESS TIMEOUT !!!
Exited with code: 0xF291

Re-ran those two after that queue finished and they encoded fine.

Is there a way to start the binary with a flag to disable the startup ‘Your version is too old’ check and dialog? If so, any guesses as to the version(s) to support such capability? I wasn’t having these kinds of problems with queues over 1000 with previous versions. Though I’d bet the queue size isn’t the problem, it just increases the likelihood of a problem in any given job.

Is there a way to have the debug log sent to a file as well as to the console? Is there a way to retain encode logs? All this info seems to be gone after a crash.

Also, the ‘Output Directory’ tab is still quite slow to respond. Changing to that tab is quicker than before, but you have to wait a bit (on the order of 60 seconds or so) before you can do anything or even switch to another tab.

Lastly, my wish list of future features:
- Default for drag-&-drop should be recursive (maybe an option setting?).
- An option to designate default action when encoding: whether to overwrite, create a new file, or query the user.
- After adding folders, the message could be a little more descriptive as to the count of files rejected for specific reasons (not an audio file, corrupted file).
- Option for more columns in the main pre-processing window (file type, size?) and sortability
- Ability to sort by status in the Processing window

Again, thanks for all your effort and dedication.

Peace, Namaste, etc.
phideaux3 is offline   Reply With Quote
Old 13th June 2011, 14:12   #295  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by phideaux3 View Post
First, just wanted to say how much I appreciate your having created this tool and continuing to maintain it. I've been using it for a few years and think it's got to be the best for my needs.

In an earlier comment, you said “try again with Build #560, which is now available via auto-update” – not sure what you meant by this. Should I be able to run ‘Check for Updates’ from the app and it should grab the new build? If so, it didn’t work. It kept saying 418 was the latest.
Currently "final" builds (such as v4.01 Final, Build #418) will only check for new stable releases. Only "beta" (pre-release) builds are able to check for new "beta" releases. So if you are currently using a "stable" version, you will have to update to a later "pre-releae" build manually, before the Auto-Update mechanism will find even newer "pre-release" builds. However the upcoming v4.02 stable release version will contain an option to check for "beta" (pre-release) updates - this option will be disabled by default.

Quote:
Originally Posted by phideaux3 View Post
Anyway, the reason I came to this forum was because 418 kept crashing my system (W7Usp1 64-bit). I was encoding to mp3 it on a quad-core with a queue of over 2k files. I reduced the threads to 3, and it didn’t crash, but over 50% of the encodes failed with error code 3 or 5 and a very small or zero-byte mp3 file.

I installed Build 574 and reduced the queue to fewer than 600 and left threads at 3. The app died with “Unhandled exception error, application will exit.” after 313 files completed.
It's hard to tell what exactly caused the crash. Do you know how to make a "debug" build and run it in a debugger?

Also: Did you watch the memory usage (Private Bytes) of the LameXP process (e.g. in ProcessExplorer). Did it increase to ~2 GB just before the crash?

[UPDATE] I just ran an encode with 800+ files on my system (Win7 x64, Core2 Quad, 4 GB of RAM, 4 threads) and everything ran through just fine.

Also the memory usage (Private Bytes) ware relatively constant at around ~55 MB. So this definitely is not a 'memory leak' problem! [/UPDATE]

Quote:
Originally Posted by phideaux3 View Post
I ran another queue of about 250 files, all finished except two, which failed with

PROCESS TIMEOUT !!!
Exited with code: 0xF291

Re-ran those two after that queue finished and they encoded fine.
Process timeout means that one of the encoder/decoder processes didn't complete (or at least send an update message) in time.

This can happen when the process encounters a deadlock. But it can also happen if the process didn't get enough CPU time by the OS to do its job.

Did you run other CPU intensive applications in the background ???

Quote:
Originally Posted by phideaux3 View Post
Is there a way to start the binary with a flag to disable the startup ‘Your version is too old’ check and dialog? If so, any guesses as to the version(s) to support such capability? I wasn’t having these kinds of problems with queues over 1000 with previous versions. Though I’d bet the queue size isn’t the problem, it just increases the likelihood of a problem in any given job.
With "previous versions" you mean v4.00 or v3.xx? Actually v4.xx is a completely new application, compared to v3.xx.

Also you can disable the update-reminder in the options menu. However extremely old versions will start to "force" the update at some point.

This can't be skipped (and usually shouldn't be skipped), except by compiling LameXP from the sources yourself and removing the check.

It is discouraged to use v3.xx nowadays, because there were a lot of nasty limitations (the lack of Unicode support is probably the worst of them).

But after all this is OpenSource software. So you can always modify the application according to your needs...

Quote:
Originally Posted by phideaux3 View Post
Is there a way to have the debug log sent to a file as well as to the console? Is there a way to retain encode logs? All this info seems to be gone after a crash.
Not currently. You can Copy&Paste the text from the console or the encode log and then save it to file (e.g. with Notepad).

But you are right. This won't be possible if the application crashed - which usually shouldn't happen

Quote:
Originally Posted by phideaux3 View Post
Also, the ‘Output Directory’ tab is still quite slow to respond. Changing to that tab is quicker than before, but you have to wait a bit (on the order of 60 seconds or so) before you can do anything or even switch to another tab.
This has been discussed before. For me it generally takes less than ~5 seconds.

But that probably depends on the "size" (number of files and sub-directories) of the initial output directory.

And of course slow Anti-Virus software might slow things down a lot here! Try to temporarily disable your real-time Anti-Virus scanner - just for testing.

There's not much I can do to make it faster, as the QFileSystemModel is a part of Qt. Already tried a number of workarounds to make it a bit faster.

I once made a test build with the Tech-Preview of Qt v4.8, which might (or might not) give some improvement. Did you check that one out?

Quote:
Originally Posted by phideaux3 View Post
Lastly, my wish list of future features:
(1) Default for drag-&-drop should be recursive (maybe an option setting?).
(2) An option to designate default action when encoding: whether to overwrite, create a new file, or query the user.
(3) After adding folders, the message could be a little more descriptive as to the count of files rejected for specific reasons (not an audio file, corrupted file).
(4) Option for more columns in the main pre-processing window (file type, size?) and sortability
(5) Ability to sort by status in the Processing window
(1) Default definitely shouldn't be recursive, I think. That's because recursively adding a directory tree might add a HUGE number of files and take a very long time, which probably isn't intended by the user. But making this an option sounds like a feasible idea.

(2) Asking the user every time the output file already exists doesn't fit very well into the "batch processing" concept of LameXP. However adding an "overwrite existing files" option would be possible, but dangerous. Maybe when I implement an option to re-name the output files.

(3) Well, how do we know what is the reason? If the type of the file couldn't be determined, is this because it's a file type we simply don't know/support or is this a broken file? It's impossible to know. After all we can only reject everything that doesn't seem proper...

(4) Not quite sure. The "size" of the input file currently is not detected (although it would be easy to add). And "file type" is a bit vague (container format? audio format? detailed compression info?). But adding some "sort" options (at least 'by filename' and 'by title') to the Source Files tab is on my TODO list.

(5) Not currently possible, as jobs are appended to the list as they are created. [EDIT] Might be easier to do than I thought at first. [/EDIT]
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 14th June 2011 at 00:46.
LoRd_MuldeR is offline   Reply With Quote
Old 13th June 2011, 20:18   #296  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
LameXP v4.02 RC-4:

An attempt to make LameXP deal better with a large number of files (will only show the most recent 50 files in the progress view now).
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 14th June 2011 at 18:52.
LoRd_MuldeR is offline   Reply With Quote
Old 13th June 2011, 23:25   #297  |  Link
phideaux3
Registered User
 
Join Date: Dec 2002
Location: NYC
Posts: 3
Thanks for the quick and informative reply.

Quote:
Originally Posted by LoRd_MuldeR View Post
Currently "final" builds (such as v4.01 Final, Build #418) will only check for new stable releases. Only "beta" (pre-release) builds will are able check for new "beta" releases.
Of course. Makes sense.

Quote:
Originally Posted by LoRd_MuldeR View Post
It's hard to tell what exactly caused the crash. Do you know how to make a "debug" build and run it in a debugger?
Not really. Unless you count shell scripting, I haven't coded since before the introduction of WNT. If there's a doc on how to do it you could point me to, I might try.

Quote:
Originally Posted by LoRd_MuldeR View Post
Also: Did you watch the memory usage (Private Bytes) of the LameXP process (e.g. in ProcessExplorer). Did it increase to ~2 GB just before the crash?

[UPDATE] I just ran an encode with 800+ files on my system (Win7 x64, Core2 Quad, 4 GB of RAM, 4 threads) and everything ran through just fine.

Also the memory usage (Private Bytes) ware relatively constant at around ~55 MB. So this definitely is not a 'memory leak' problem! [/UPDATE]
Didn't notice, but your assessment sounds accurate. I was watching Memory Usage in Task Manager, it seemed to hover just over 2G for the entire system, but I wasn't watching right before the crash.

Quote:
Originally Posted by LoRd_MuldeR View Post
Process timeout means that one of the encoder/decoder processes didn't complete (or at least send an update message) in time.

This can happen when the process encounters a deadlock. But it can also happen if the process didn't get enough CPU time by the OS to do its job.

Did you run other CPU intensive applications in the background ???
Not that I can think of. Firefox? Outlook? CPU performance was floating just under 100% with three threads.

I've been running some more tests, and I was getting the PROCESS TIMEOUT quite often -- as early as the 14th and 20th file in the queue. (Private RAM approaches but never gets to 37MB.) I decreased the threads to two, and over 100 files have now completed without problem. The odd thing to me is that this never happened w/ the previous version (I think I was using the Xmas build on this same system for quite a while.)

Quote:
Originally Posted by LoRd_MuldeR View Post
(1) Default definitely shouldn't be recursive, I think. That's because recursively adding a directory tree might add a HUGE number of files and take a very long time, which probably isn't intended by the user. But making this an option sounds like a feasible idea.
Not making it the default makes sense -- as long as it's an option.

Quote:
Originally Posted by LoRd_MuldeR View Post
(3) Well, how do we know what is the reason? If the type of the file couldn't be determined, is this because it's a file type we simply don't know/support or is this a broken file? It's impossible to know. After all we can only reject everything that doesn't seem proper...
I guess I was supposing that you could ignore files by extension. Doesn't seem to make sense to attempt to load an md5, for instance. Maybe this functionality could be an option -- "Loadable file types"?

Quote:
Originally Posted by LoRd_MuldeR View Post
(4) Not quite sure. The "size" of the input file currently is not detected (although it would be easy to add). And "file type" is a bit vague (container format? audio format? detailed compression info?). But adding some "sort" options (at least 'by filename' and 'by title') to the Source Files tab is on my TODO list.
Cool. Re: "file type", same supposition as before. My goal would be that if I toss in some folders which already have some mp3's in them, I'd like to not waste the time re-encoding them.

I forgot another wish list item, which would be to be able to select multiple items in the list (shift and/or ctrl keys) and remove all selected. Am I correct that currently it's only possible to remove one at a time?

Quote:
Originally Posted by LoRd_MuldeR View Post
(5) Not currently possible, as jobs are appended to the list as they are created. [EDIT] Might be easier to do than I thought at first. [/EDIT]
I think you can see the value in being able to sort the results and have all the failures grouped together.

Thanks again!!!
phideaux3 is offline   Reply With Quote
Old 14th June 2011, 00:05   #298  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by phideaux3 View Post
Not really. Unless you count shell scripting, I haven't coded since before the introduction of WNT. If there's a doc on how to do it you could point me to, I might try.
You'd have to install Visual Studio (I think one of the free Express editions does the job), compile yourself a "Debug" build and start debugging. This way it will switch over to the debugger on crash (and show you the line in code where it crashed) instead of just throwing an error message. If you never used an IDE, like Visual Studio, before, then it will need some familiarization though. Depends on how much time you are willing to spend. Note that there are some basic compile instructions in the LameXP FAQ document (you will need to install Qt as well).

Quote:
Originally Posted by phideaux3 View Post
Didn't notice, but your assessment sounds accurate. I was watching Memory Usage in Task Manager, it seemed to hover just over 2G for the entire system, but I wasn't watching right before the crash.
Well, the global memory usage doesn't say much. Even if all physical RAM is in use, the OS can still use the swap file on the HDD to allocate more memory.

If there is some problem with memory, then it's the 2 GB per process memory limit. A single 32-Bit process cannot allocate more then 2 GB of memory, no matter what.

But it doesn't seem to be a problem with LameXP. Even when processing hundreds of files at once, memory usage for the LameXP process stays below ~60 MB.

Quote:
Originally Posted by phideaux3 View Post
Not that I can think of. Firefox? Outlook? CPU performance was floating just under 100% with three threads.
Well, the goal is to have ~100% CPU usage, i.e. not waste any CPU cycles idle.

That's why we create/run several threads in parallel. If CPU usage drops below 100%, it means the CPU or at least some of its cores were idle some of the time.

If, however, there are some (high priority) processes that "eat" too many CPU cycles, it can happen that LameXP's encoder processes starve and eventually trigger a timeout.

Note that LameXP's encoder processes are running with reduced priority. This is done in order to keep the system responsive while encoding...

Quote:
Originally Posted by phideaux3 View Post
I've been running some more tests, and I was getting the PROCESS TIMEOUT quite often -- as early as the 14th and 20th file in the queue. (Private RAM approaches but never gets to 37MB.) I decreased the threads to two, and over 100 files have now completed without problem. The odd thing to me is that this never happened w/ the previous version (I think I was using the Xmas build on this same system for quite a while.)
That is really strange. I never get that. Not one single time. And you are the first user to complain about this issue.

I'm not quite sure what I can do, except for increasing the timeout interval. But it already is 30 seconds.

If the encoder process doesn't respond (i.e. either complete or send a progress update) within 30 seconds, something has to be seriously wrong...

Also with the old 3.xx versions you couldn't get a timeout error, simply because there wasn't any timeout check implemented

(If one of the encoder processed deadlocked, LameXP v3.xx would have waited until forever...)

Quote:
Originally Posted by phideaux3 View Post
I guess I was supposing that you could ignore files by extension. Doesn't seem to make sense to attempt to load an md5, for instance. Maybe this functionality could be an option -- "Loadable file types"?
No, we cannot use file extensions. File extensions are neither unambiguous nor reliable. Guessing the file's type form its "extensions" is the most stupid way to implement this.

LameXP does it properly: It uses MediaInfo to actually analyze the file and detect the file's type from it's content. The file's name (including its extension) is not considered at all.

Quote:
Originally Posted by phideaux3 View Post
I forgot another wish list item, which would be to be able to select multiple items in the list (shift and/or ctrl keys) and remove all selected. Am I correct that currently it's only possible to remove one at a time?
I will look for a solution.

Quote:
Originally Posted by phideaux3 View Post
I think you can see the value in being able to sort the results and have all the failures grouped together.
Yes. If you encoded a huge number of files and only a few failed, it will be hard to find those on the list...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 14th June 2011 at 01:42.
LoRd_MuldeR is offline   Reply With Quote
Old 14th June 2011, 08:15   #299  |  Link
phideaux3
Registered User
 
Join Date: Dec 2002
Location: NYC
Posts: 3
Ran another set of files with only two threads.

2 of 1587 files failed with

PROCESS TIMEOUT !!!
Exited with code: 0xF291

One failed running tool_lame.exe, one didn't get past tool_flac.exe.

I can only guess from that low-prio jobs are getting pushed back at peak times and having to wait longer than the timeout. Any chance of making the timeout another option? Maybe a range of up to 5 minutes? It would seem that when I used the older version, nothing (thanfully) ever deadlocked, but because there was no timeout, nothing ever failed for that reason.

PS - max private RAM consumed with that queue was <53MB.

phideaux3 is offline   Reply With Quote
Old 14th June 2011, 12:03   #300  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Still, unless you are running other CPU intensive tasks in parallel to LameXP there shouldn't be any problem. And even if you are running CPU intensive processes in parallel to LameXP and these processes have an increased priority, then the OS' scheduler shouldn't let the encoder processes starve. They might get less CPU time than the higher priority processes, yes. They might have to wait a few seconds now and then, yes. But more than 30 seconds? That's definitely too much. The only situation in which a process (one that is not in the "idle" priority class) might starve is when there's another processes running with "real time" priority and that processes uses the CPU all the time. But that's the reason why using the "real time" priority class is highly discouraged - especially for CPU intensive tasks. So either there is something seriously wrong with your system (did you install any CPU "tweak" or "optimization" tools?) or there's another problem that we didn't consider so far. Last but not least: Did you try to disable the real-time scanner (guard) of your Anti-Vrius software? It wouldn't be the first time that it turns out an Anti-Virus scanner caused all the trouble...

(Anyway, I will increase the timeout to 3 minutes. I don't want to have yet another option. Will send you the new build via PM soon!)
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 14th June 2011 at 17:45.
LoRd_MuldeR is offline   Reply With Quote
Reply

Tags
aac, aotuv, flac, lame, lamexp, mp3, mp4, ogg, oggenc, opus, vorbis

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.