Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Programming and Hacking > Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th August 2017, 18:32   #1461  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
Quote:
Originally Posted by cindernat View Post
First of all, great program - thank you so much.

One issue I'm having is when I am doing a batch conversion, for any folder that has files in its root directory plus additional folders, the additional folders are ignored.

EG: Within Artist A's folder, I have 3 albums and 4 single tracks (single tracks existing in that artist's root directory). In this scenario, only the single tracks are converted and the 3 album folders are skipped.

Yet Artist B, who only has album folders within their folder and no single tracks, nothing is skipped and all converted.

I hope that makes sense.

Anyone else had this occur?
You mean this happens when you are adding a directory to the "Source Files" tab via Drag'n'Drop?

The reason is that, in this case, only the files in the directory itself are added. We do not, by default, add the whole directory tree recursively, because it could easily take ages on a massive directory tree. The only case when it will descend into sub-directories automatically is when the "starting" directory did not contain any files at all. And even then, it only descends to the next level, until some file have been found.

If you which to add a whole directory tree, in a fully recursive manner, you can do so via "File" -> "Open Folder Recursively".
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 30th August 2017 at 18:37.
LoRd_MuldeR is offline   Reply With Quote
Old 31st August 2017, 07:12   #1462  |  Link
cindernat
Registered User
 
Join Date: Aug 2017
Posts: 2
Quote:
Originally Posted by LoRd_MuldeR View Post
You mean this happens when you are adding a directory to the "Source Files" tab via Drag'n'Drop?

The reason is that, in this case, only the files in the directory itself are added. We do not, by default, add the whole directory tree recursively, because it could easily take ages on a massive directory tree. The only case when it will descend into sub-directories automatically is when the "starting" directory did not contain any files at all. And even then, it only descends to the next level, until some file have been found.

If you which to add a whole directory tree, in a fully recursive manner, you can do so via "File" -> "Open Folder Recursively".
Yep - that'd be it then. Didn't realise! Will take note for next time. Thank you!
cindernat is offline   Reply With Quote
Old 15th September 2017, 21:57   #1463  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
LameXP v4.16 Alpha-4
https://sourceforge.net/projects/lam...6.exe/download

Quote:
Changes between v4.15 and v4.16 [*unreleased*]:
* Upgraded build environment to Microsoft Visual Studio 2017 with Update-3
* Updated Opus encoder/decoder libraries to v1.2.1 (2017-07-26) and Opus-Tools to v0.1.10 (2017-05-25)
* Updated MediaInfo to v0.7.98 (2017-08-08), compiled with ICL 17.0 and MSVC 12.0
* Some improvements to output file name generation code
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 6th October 2017, 08:38   #1464  |  Link
dissory
Registered User
 
Join Date: Mar 2017
Posts: 13
Thanks for the great program, LoRd_MuldeR!

One thing I noticed though is if I convert .wav files containing metadata to .flac and back again to .wav all the original .wav metadata such as "Track name" is lost. Any way to fix this?
dissory is offline   Reply With Quote
Old 6th October 2017, 18:47   #1465  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
Quote:
Originally Posted by dissory View Post
Thanks for the great program, LoRd_MuldeR!

One thing I noticed though is if I convert .wav files containing metadata to .flac and back again to .wav all the original .wav metadata such as "Track name" is lost. Any way to fix this?
Hello.

The procedure is as follows: We use MediaInfo to detect the meta tags from the original input file, then we decompress the input file to WAV, and finally we encode to temporary WAV using the selected encoder (e.g. LAME, VorbisEnc, OpusEnc, or FLAC). Also, when we invoke the selected encoder, we forward the previously detected meta tags - as far as possible. The capabilities of the different encoders supported by LameXP differ greatly in what meta tags they support. But most of them support at least the bear minimum. Now, the "problem" with the WAV format is that WAV format does not really use an encoder at all! WAV format is the uncompressed output of the decoder. If, for example, you "convert" from FLAC to WAV, the resulting WAV file is actually the direct output of the FLAC decoder! There is no "encoding" step, in which we could embed the meta tags. And, apparently, the FLAC decoder does not add meta tags to the WAV file it produces.

BTW: I'm not even sure there is a standardized way of storing meta tags in WAV files. Sure, the WAV format is based on the RIFF format, so you can add arbitrary additional RIFF chunks to the file and any well-behaving application should simply ignore those additional RIFF chunks, if it doesn't understand them. But unless there is a widely-support standard for embedding meta tags in WAV format, it will be pretty pointless endeavor...
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 6th October 2017 at 19:32.
LoRd_MuldeR is offline   Reply With Quote
Old 8th October 2017, 13:33   #1466  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 23
hello,
just wondering: why is multithreading limited to 10 threads?
is it on purpose. is there a setting somewhere?
edit: nevermind. i just found the setting. seems like autodetection is not working properly. setting it manually fixes it

Last edited by RieGo; 8th October 2017 at 13:36.
RieGo is offline   Reply With Quote
Old 8th October 2017, 14:12   #1467  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
Quote:
Originally Posted by RieGo View Post
hello,
just wondering: why is multithreading limited to 10 threads?
is it on purpose. is there a setting somewhere?
edit: nevermind. i just found the setting. seems like autodetection is not working properly. setting it manually fixes it
"Auto-detection" is working as expected, probably. That is because auto-detection has not been implemented to simply run as many encoder instances in parallel as there are (logical) CPU cores available. Instead, it has been optimized to run as many encoder instances in parallel as give the maximum throughput (or, equivalently, minimize overall encoding time). Note that running too many instances in parallel can easily result in HDD thrashing and thus slow things down considerably, as all those parallel instances also need to read their respective source file and write their respective output file. The following "cores to instances" curve has been derived experimentally, with the help of various LameXP users:

https://i.imgur.com/6XYwr03.png
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 8th October 2017 at 14:17.
LoRd_MuldeR is offline   Reply With Quote
Old 8th October 2017, 15:09   #1468  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 23
Quote:
Originally Posted by LoRd_MuldeR View Post
"Auto-detection" is working as expected, probably. That is because auto-detection has not been implemented to simply run as many encoder instances in parallel as there are (logical) CPU cores available. Instead, it has been optimized to run as many encoder instances in parallel as give the maximum throughput (or, equivalently, minimize overall encoding time). Note that running too many instances in parallel can easily result in HDD thrashing and thus slow things down considerably, as all those parallel instances also need to read their respective source file and write their respective output file. The following "cores to instances" curve has been derived experimentally, with the help of various LameXP users:

https://i.imgur.com/6XYwr03.png
wow. that's interesting. thanks for clearing up.
in my case it was faster using more threads manually, but that doesn't mean it's the same for everybody. probably depends on a lot of components.
RieGo is offline   Reply With Quote
Old 8th October 2017, 16:14   #1469  |  Link
mariush
Registered User
 
Join Date: Dec 2008
Posts: 557
That may have to revisited, or may need to use a number of threads if the input and outputs are on mechanical drives, and another number for SSDs , RAMDisks, NAS etc.
mariush is offline   Reply With Quote
Old 8th October 2017, 16:17   #1470  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
In fact, we should probably use a simple "1:1" mapping, when a SSD (solid-state drive) is detected. But it turns out to be surprisingly difficult to tell SSD's and "normal" HDD's apart on windows

Here is an attempt (simple command-line application for now), maybe you can give it a try:
https://sourceforge.net/projects/mul...8.zip/download
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 8th October 2017 at 16:22.
LoRd_MuldeR is offline   Reply With Quote
Old 8th October 2017, 16:35   #1471  |  Link
mariush
Registered User
 
Join Date: Dec 2008
Posts: 557
Yeah, it works fine for me.
First thought that came to mind was running a simple test like reading some 10K random 512 bytes sector reads from the hard drive, as such a big quantity of sectors and so spread out would be unlikely to be cached by the OS. Mechanical drives (even hdd with some nand cache) would have high seek times, while SSDs and ram drives would be fast. NAS drives may be somewhere in between, due to tcp/ip and all that.
mariush is offline   Reply With Quote
Old 8th October 2017, 18:49   #1472  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 23
Quote:
Originally Posted by LoRd_MuldeR View Post
In fact, we should probably use a simple "1:1" mapping, when a SSD (solid-state drive) is detected. But it turns out to be surprisingly difficult to tell SSD's and "normal" HDD's apart on windows

Here is an attempt (simple command-line application for now), maybe you can give it a try:
https://sourceforge.net/projects/mul...8.zip/download
my 2 ssds are correctly detected, but the encrypted volume that's on an ssd isn't detected at all (maybe because it's mounted as removable storage?)
but still that's a good starting point

Last edited by RieGo; 8th October 2017 at 19:05.
RieGo is offline   Reply With Quote
Old 8th October 2017, 20:04   #1473  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
Quote:
Originally Posted by RieGo View Post
my 2 ssds are correctly detected, but the encrypted volume that's on an ssd isn't detected at all (maybe because it's mounted as removable storage?)
Not much of a surprise. What we can do is mapping a logical drive (i.e. drive letter) to the underlying physical device or devices – multiple logical drives may map to the same physical device, and a logical drive may span multiple physical devices. Then we can query certain properties of those physical device(s) which allow us to "detected" an SSD or SSD-like drive. However, if some software creates a "virtual" drive, it is pretty much impossible to tell on which physical drive or drives the data is actually stored. It may emulate (and thus look like) a "fixed" disk, but we don't know whether the contents of "virtual" drive are stored on a (local) physical device at all – might be a "RAM" disk, or a "cloud" drive, or whatsoever...
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 8th October 2017 at 20:12.
LoRd_MuldeR is offline   Reply With Quote
Old 13th October 2017, 20:18   #1474  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
LameXP v4.16 Alpha-5
https://sourceforge.net/projects/lam...2.exe/download

Quote:
Changes between v4.15 and v4.16 [*unreleased*]:
* Upgraded build environment to Microsoft Visual Studio 2017 with Update-4
* Updated Opus encoder/decoder libraries to v1.2.1 (2017-07-26) and Opus-Tools to v0.1.10 (2017-05-25)
* Updated MediaInfo to v0.7.98 (2017-08-08), compiled with ICL 17.0 and MSVC 12.0
* Updated GnuPG to v1.4.22 (2017-07-19), compiled with GCC 7.2.0
* Improved auto-detection of max. parallel instances on computers with SSD (or similar) drive
* Some improvements to output file name generation code
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 17th October 2017, 15:16   #1475  |  Link
Motenai Yoda
Registered User
 
Motenai Yoda's Avatar
 
Join Date: Jan 2010
Posts: 632
It's me or when there is a beta update it can't be avoided?
__________________
powered by Google Translator
Motenai Yoda is offline   Reply With Quote
Old 17th October 2017, 20:10   #1476  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,799
Quote:
Originally Posted by Motenai Yoda View Post
It's me or when there is a beta update it can't be avoided?
See information included with pre-release ("beta") builds:

Code:
LameXP Pre-Release Version
--------------------------

This pre-release version of LameXP is intended for testing and evaluation only.
We recommend to *not* use this version for production. Use it at your own risk!

Note: This "test" version of LameXP will expire 30 days after the release date.
LameXP is free/libre software. The "stable" versions of LameXP do *not* expire!
So yeah, if you want to support development of LameXP by testing out pre-release ("beta") builds, then you'll need to update to the latest build about every ~30 days.

There is nothing, though, that would urge you to upgrade from the latest "stable" version to any "beta" version...
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 17th October 2017 at 20:19.
LoRd_MuldeR is offline   Reply With Quote
Reply

Tags
aac, aotuv, flac, lame, lamexp, mp3, mp4, ogg, oggenc, opus, vorbis

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.