L-SMASH Source

[LigH]

I remember that there was recently an update to at least ignore Atmos extensions gracefully ... but which software was it?

Ah: LAV Filters 0.63.0 can ignore it now (http://forum.doom9.org/showthread.php?p=1695625#post1695625); eac3to may still need an update.

So if LAV Filters can, I wondered if LwLibav could as well. But that might require to port the patch? In the meantime, DDS2Mod might work for AviSynth users.

[the_weirdo]

I've uploaded new builds that linked with Libav libs. Those builds seem to work with files that contain TrueHD+Atmos tracks (though they can't decode those TrueHD+Atmos tracks at all).

EDIT: My bad. Those new builds are broken so I have to pull them down and do more testings before uploading new builds.
EDIT2: As it turned out there's no problem with FFmpeg builds (compare to Libav builds), I won't update Libav builds anymore.

There are some misunderstandings caused by my poor English and bad testing method. So to make thing clearer, builds linked with FFmpeg should be able to decode TrueHD from TrueHD+Atmos tracks (they work with some samples I have) while builds linked with Libav can't decode TrueHD+Atmos tracks at all. As Libav builds now don't offer any advantage over FFmpeg builds, I won't update them anymore.

[LigH]

:thanks:

Ignoring Atmos extensions is fine. As far as I read, their "rendering" is speaker setup dependent... They just should not make demuxing and decoding TrueHD audio tracks fail.

[LigH]

OK, the ffmpeg based L-SMASH Source build r733 was able to load the sample I received; now I hope the other user can handle the whole audio.

[forclip]

LSMASHSource (tested with L-SMASH-Works-r775-20150108-32bit from here (https://www.dropbox.com/sh/3i81ttxf028m1eh/AAABkQn4Y5w1k-toVhYLasmwa?dl=0)) have a memory leak: when indexing unsupported video files, it doesn't freed the memory of the process, and it seems like the whole file is readed into it.
For example

ffmpeg -i "input_file.avs" -vcodec rawvideo -pix_fmt bgra64le -an "output_file.avi"
where "input_file" is a simple AviSynth script:

ColorBars()
Trim(0, 1000)
then load resulting AVI file (2+Gb) into AviSynth (http://forum.doom9.org/showthread.php?t=148782)

LWLibavVideoSource("D:\Temp\output_file.avi")
While indexing, the memory consumption of the process (AvsP, MPC-HC,..) has increased to the same 2+Gb, and after that

LWLibavVideoSource: failed to get the video track.

Unloading the script doesn't freed the memory.

[stax76]

@VFR maniac

I have a small request:

if this is possible from a avs filter, please write indexing progress to stdout.

[LigH]

May be complicated when it is not connected to any console. But possibly via a debug message API?

[stax76]

In StaxRip I run DGIndex exactly like I run almost all command line apps and it receives progress from DGIndex so it must be in the source code and neuron2 must know it.

[qyot27]

But you're still using a separate CLI program that outputs its progress to stdout to do the indexing; DGDecode.dll isn't what powers the indexing in DGIndex. You'd need to look at FFMS2 and how it interacts with ffmsindex to see how to implement this for LSMASHSource.dll, provided there's a similar reporting structure in place in the code. Or just take the suggestion from the corresponding Github issue (https://github.com/VFR-maniac/L-SMASH-Works/issues/17) and use aui_indexer to generate the indices for LSMASHSource's filters rather than the .dll itself. I don't know if aui_indexer prints its progress or not (and if you want to support XP and pre-SSE2 users, you'll need to recompile it from the provided source).

[Selur]

don't know if aui_indexer prints its progress or not
it doesn't, posted about this at the beginning of this thread (https://www.doom9.org/showthread.php?p=1620445#post1620445) which is when I created the issue :)

[VFR maniac]

Write like the plugin for AviUtl.
I rarely care about the plugin for AviSynth.
I'm busy and lazy.

Hints:
https://github.com/VFR-maniac/L-SMASH-Works/blob/master/AviUtl/progress_dlg.c
https://github.com/VFR-maniac/L-SMASH-Works/blob/master/AviUtl/lwlibav_input.c#L104
https://github.com/VFR-maniac/L-SMASH-Works/blob/master/AviSynth/lwlibav_source.h#L24
https://github.com/VFR-maniac/L-SMASH-Works/blob/master/AviSynth/lwlibav_source.cpp#L74

[speedyrazor]

Hi, I am opening very large HD Quicktime Prores files, about 120GB, with LSMASHVideoSource, as I don't want to wait 20 mins for an index.
I am finding using the below avisynth script that it is still taking about 1-2 mins to open my file.
My source quicktime contains 8 channels of audio, which I need to use.
My question is... Is there anything I can do to speed up the opening of my files?

v = LSMASHVideoSource(source="E:\Source22.mov", format="YUY2")
a1 = LSMASHAudioSource(source="E:\Source22.mov", track=2)
a2 = LSMASHAudioSource(source="E:\Source22.mov", track=3)
a3 = LSMASHAudioSource(source="E:\Source22.mov", track=4)
a4 = LSMASHAudioSource(source="E:\Source22.mov", track=5)
a5 = LSMASHAudioSource(source="E:\Source22.mov", track=6)
a6 = LSMASHAudioSource(source="E:\Source22.mov", track=7)
a7 = LSMASHAudioSource(source="E:\Source22.mov", track=8)
a = MergeChannels(a1,a2,a3,a4,a5,a6,a7)
AudioDub(a,v)
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)
Spline64Resize(720, 576)
AssumeFPS(25, 1)

Kind regards.

[speedyrazor]

Hi, When using LSMASHAudioSource with 24 bit audio all I am getting is white noise. Is LSMASHAudioSource compatible with 24 bit audio?

Kind regards.

[Selur]

Not atm. (LWLibavAudioSource is, but that requires an index)

[VFR maniac]

Hi, When using LSMASHAudioSource with 24 bit audio all I am getting is white noise. Is LSMASHAudioSource compatible with 24 bit audio?

Kind regards.

Yes, if the CODEC and its configuration are correctly detected.
If not, send such sample please.

[VFR maniac]

Hi, I am opening very large HD Quicktime Prores files, about 120GB, with LSMASHVideoSource, as I don't want to wait 20 mins for an index.
I am finding using the below avisynth script that it is still taking about 1-2 mins to open my file.
My source quicktime contains 8 channels of audio, which I need to use.
My question is... Is there anything I can do to speed up the opening of my files?



Kind regards.

Because the internal indexing occurs per each clip (video and individual audio channels), I guess.
To solve this, we need an implementation like the following method.

clip = LSMASHSource(source="source.mov")
v = LSMASHGetTrack(clip, track=1)
a1 = LSMASHGetTrack(clip, track=2)
a2 = LSMASHGetTrack(clip, track=3)
a3 = LSMASHGetTrack(clip, track=4)
a4 = LSMASHGetTrack(clip, track=5)
a5 = LSMASHGetTrack(clip, track=6)
a6 = LSMASHGetTrack(clip, track=7)
a7 = LSMASHGetTrack(clip, track=8)
a = MergeChannels(a1,a2,a3,a4,a5,a6,a7)
AudioDub(a,v)


I don't know if it can be achieved within AviSynth API restrictions.

[Selur]

Is there anything I can do to speed up the opening of my files?
you could create the index before opening the file,...
Just out of curiosity how large is the index file?

[speedyrazor]

Yes, if the CODEC and its configuration are correctly detected.
If not, send such sample please.

Hi, thanks for you response. I have sent you a PM with a link to a small test file.
I have tried using various source videos, all prores, all with 24 bit audio, I have also tried various settings, but all come out with white noise (same files with 16 bit audio work fine).
The sample I sent you is a typical prores file with multichannel 24 bit audio.
Thanks for your help.

Kind regards.

[speedyrazor]

you could create the index before opening the file,...
Just out of curiosity how large is the index file?

OK, so using only LWLibavVideoSource for a 90 min HD Prores at 113 GB, it took 12 mins to index and produced a 457 MB file.
This was using a 10 GB network with media which was on a network share.
Here is the AVS script I used.

LWLibavVideoSource(source="J:\Folder\test.mov", format="YUY2")

Kind regards.

[VFR maniac]

Hi, thanks for you response. I have sent you a PM with a link to a small test file.
I have tried using various source videos, all prores, all with 24 bit audio, I have also tried various settings, but all come out with white noise (same files with 16 bit audio work fine).
The sample I sent you is a typical prores file with multichannel 24 bit audio.
Thanks for your help.

Kind regards.

Should be fixed @ L-SMASH rev1339 + L-SMASH Works rev782
Thank you for sending a sample.

[speedyrazor]

Hi VFR maniac, you are a star!!!! Many thanks for sorting this out so quickly.
Sorry to appear stupid, but where can I download the LSMASHSource.dll from?

Kind regards.

[LigH]

Let's hope the_weirdo will publish builds (http://forum.doom9.org/showthread.php?p=1689573#post1689573) soon.

[speedyrazor]

And here it is.....
Many thanks to VFR maniac and the_weirdo.

https://www.dropbox.com/sh/3i81ttxf028m1eh/AAABkQn4Y5w1k-toVhYLasmwa?dl=0

And I can report that all is now working as it should with 24 bit audio.
Thanks againVFR maniac.

[speedyrazor]

Because the internal indexing occurs per each clip (video and individual audio channels), I guess.
To solve this, we need an implementation like the following method.

clip = LSMASHSource(source="source.mov")
v = LSMASHGetTrack(clip, track=1)
a1 = LSMASHGetTrack(clip, track=2)
a2 = LSMASHGetTrack(clip, track=3)
a3 = LSMASHGetTrack(clip, track=4)
a4 = LSMASHGetTrack(clip, track=5)
a5 = LSMASHGetTrack(clip, track=6)
a6 = LSMASHGetTrack(clip, track=7)
a7 = LSMASHGetTrack(clip, track=8)
a = MergeChannels(a1,a2,a3,a4,a5,a6,a7)
AudioDub(a,v)


I don't know if it can be achieved within AviSynth API restrictions.

Could something like cache bool = true be implemented, so it only looks at it once?

Kind regards.

[RTW47]

since there is no cache parameter for LSMASH*Source does this mean that index always is stored inside memory and discarded after the script is closed? (just like LWLibav*Source/FFMS2 with cache=false would?) because there is still some internal indexing involved, if i understood correctly. (more interested in vapoursynth version if there are any differences)

[speedyrazor]

Hi, I am trying to open a Quicktime Prores 444 file with LSMASHVideoSource but just get the below error:

[avisynth @ 029017e0] Avisynth: access violation at 0x0076F2E1 in C:\Program Files (x86)\AviSynth 2.5\plugins\LSMASHSource.dll, attempting to read from 0x00000000
(C:\Temp\SC_Test.avs, line 1)
C:\Temp\SC_Test.avs: Unknown error occurred

I am using this ASV script:

LSMASHVideoSource(source="Prores_444.mov", format="YUV444P10")

I have also tried various other 'format's.
Should LSMASHVideoSource be able to open Prores 444 files?

[VFR maniac]

444? not 4444?
LSMASHVideoSource should be able to open ProRes 422 and 4444 (XQ).

[speedyrazor]

444? not 4444?
LSMASHVideoSource should be able to open ProRes 422 and 4444 (XQ).

ffmpeg reports it to be this:

Stream #0:0(eng): Video: prores (ap4h / 0x68347061), yuv444p10le(bt709), 1920x1080, 22649 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc (default)

I have PM'd you a small sample.

Kind regards.

[VFR maniac]

The 4CC 'ap4h' represents Apple ProRes 4444.

I can't reproduce your bug. Here, it's processed normally.
I have a question to you. What ver. of AviSynth do you use?
AviSynth 2.5.8 can't handle some planar pixel formats.

If you don't use such old AviSynth, I guess you encountered miscompilation, but I'm not sure
Usually, MSVC+GCC is unstable, especially, when static linking is used.
Anyway, it is difficult to investigate without debug builds.

[speedyrazor]

The 4CC 'ap4h' represents Apple ProRes 4444.

I can't reproduce your bug. Here, it's processed normally.
I have a question to you. What ver. of AviSynth do you use?
AviSynth 2.5.8 can't handle some planar pixel formats.

If you don't use such old AviSynth, I guess you encountered miscompilation, but I'm not sure
Usually, MSVC+GCC is unstable, especially, when static linking is used.
Anyway, it is difficult to investigate without debug builds.

I am very sorry VFR maniac, I tried on a different system, and it worked fine. Sorry for wasting your time. I will do a re-install on the current system.

Kind regards.

[hello_hello]

I guess this is the place to report it......

The version of L-Smash currently on MeGUI's auto-update server (r783) doesn't seem to work at all on XP. Avisynth won't load it.
http://forum.doom9.org/showthread.php?p=1710813#post1710813

I've reverted back to r768 and that works fine, although I have a question regarding the recently added variable to constant frame rate conversion. I haven't tested it with lots of sources, but should it work regardless of file type? So far I've had it working for AVIs and MP4s but it seems to have no effect when the source is MKV. Should it?

Thanks.

[sneaker_ger]

Works fine here with mkv (the_weirdo's 783). Be more specific and supply a sample.

[hello_hello]

Originally I thought it was specific to MKV because the frame rate conversion seemed to have no effect for the couple I tried but it turns out that's not correct. However it is producing odd results with some files so I'll try to split off a few small samples shortly. I can't upload large files as my internet connection is way too slow so I'll need to split some files and try to create small samples.

[Groucho2004]

Works fine here with mkv (the_weirdo's 783).
It works fine because you're not using XP, right?
I already explained what the problem is in the Megui thread. r783 is not compatible with XP.

[sneaker_ger]

I was talking about VFR->CFR with mkv files, not the XP issue.

[hello_hello]

Okay.... here's a few little samples. Each file contains exactly the same video, remuxed into different containers. The duration is just over a minute. I don't know what the deal is with this particular video but if I had to pick who to believe in respect to frame count (for a constant frame rate) and duration, I'd always put my money on AviSource.

AVI
AviSource: 1593 frames, duration 00:01:06:441, 23.976fps.
FFMS2: Same as AviSource with and without frame rate conversion.
LWLibavVideoSource: Same frame count but only 0.3fps and a duration of 01:28:35:637. Adding "fpsnum=24000, fpsden=1001" frame rate conversion increases the total frame count to over 100,000.

TS
DirectShowSource: Same result as for AviSource above (minus one frame when convertfps=true is added).
FFMS2: Shows a little FFMS2 oddness in that the frame count changes by 3 when "fpsnum=24000, fpsden=1001" is used, but it is ffms2 and a TS file.
LWLibavVideoSource: Same result as for AviSource above when "fpsnum=24000, fpsden=1001" is added to the script. Without it the duration decreases by about 11ms.

MKV
DirectShowSource: Seems to behave pretty well for DirectShowSource aside from not liking the last 3 frames.
FFMS2: Behaves fairly well although when "fpsnum=24000, fpsden=1001" is added the frame count drops by three. It appears to be ignoring the first couple of frames.
LWLibavVideoSource: Agrees with AviSource above as to the total number of frames, but not the duration. Output is 24.471fps. There's no convincing it to change the frame rate. You can add "fpsnum=96000, fpsden=1001" to a script if you like, but the output will still be 24.471fps.

Make of that what you will. ;)
I'm testing with L-Smash r768.

35MB zip file containing the video in three different containers. http://ge.tt/1PYYr0B2/v/0

As a side note, I've seen ffms do odd things with AVIs when frame rate conversion is added to the script, even when the specified frame rate is the original fame rate. That problem (http://forum.doom9.org/showthread.php?p=1709589#post1709589) has been discussed a few times. The video from which I split the samples off for this post is the only one where I've seen ffms2 do the same thing for a different file type (ie MKV) but I'm having trouble splitting off a small sample without the problem going away.

Anyway..... could someone please tell me why ffms2 and L-Smash treat AVIs with null frames as variable frame rate? Given they're plugins for AviSynth I'd think it far more logical not to. That's a big part of the motivation for using fpsnum and fpsden when opening AVI's, to make sure the output is the correct constant frame rate, just like AviSource would do it.....

[Groucho2004]

There's nothing wrong with the video.

First I demuxed the h.264 stream to determine the properties. Frame rate is 23.976, 1594 frames.

Muxed to mkv, no problem with LWLibavVideoSource reading that mkv, frame rate and number of frames are fine.

FFMS2 however needs a "assumefps(24000, 1001)" in order to behave.

[sneaker_ger]

The mkv has a video delay so demuxing will change the results. I don't have time to look into this further at the moment.

[hello_hello]

You're probably right about the total frame count being 1594. MeGUI's preview displays 1593 bit I keep forgetting about the first frame being frame zero.

You might be able to demux and remux the video to get it to behave, but what about the MKV I uploaded? Can you use fpsnum and fpsden to change the frame rate? And what about the AVI, does LWLibavVideoSource lose the plot completely as it does for me?

Anyway, I think I've had enough for today. I've been messing around with this way too long. I'll probably check back tomorrow.

[hello_hello]

The mkv has a video delay so demuxing will change the results. I don't have time to look into this further at the moment.

Ahh..... that might explain why sometimes the frame count changes by three or four, depending on the container and who's decoding, but I have no idea why there should be a delay. I remuxed the original MKV without any audio, then split it into sections until I found one that misbehaved, and it wasn't at the beginning.

See you tomorrow (tomorrow for me, at least).

[Groucho2004]

You might be able to demux and remux the video to get it to behave, but what about the MKV I uploaded? Can you use fpsnum and fpsden to change the frame rate? And what about the AVI, does LWLibavVideoSource lose the plot completely as it does for me?
The mkv and avi you uploaded are obviously bolloxed.
The elementary stream is fine and when I mux it into a mkv, everything is OK. The question is - what went wrong when you created the mkv?

Lastly, muxing avc into an avi should be avoided anyway.

[hello_hello]

The mkv and avi you uploaded are obviously bolloxed.
The elementary stream is fine and when I mux it into a mkv, everything is OK. The question is - what went wrong when you created the mkv?

Lastly, muxing avc into an avi should be avoided anyway.

The AVI is only bolloxed when using LWLibavVideoSource. I normally wouldn't put AVC in an AVI without a specific reason but I thought I'd test it.

If something went wrong with the MKV when it was created it's persisted through several remuxes and splitting and seems to only effect LWLibavVideoSource. How "something going wrong" could stop L-Smash from doing frame rate conversion is still a mystery.

If you're motivated, try remuxing the extracted stream with ffmpeg. I can't get it to play nice. It keeps dropping the first 50 seconds and I'm left with a 16 second MP4/MKV. It's odd.

I've really got to do some work. See you later......

[kolak]

Have a small issue with new ffmpeg supported format, which is GrassValey HQX codec.
Height is returned in vs as 1088 (instead of 1080), so I assume it misses some automatic cropping (looks like LWLibavSource forces mod16 mode).
I can provide sample file if needed.

[hello_hello]

Is L-Smash deliberately snubbing XP these days? The latest version (r784) won't run on XP. Neither would r783.

[Reel.Deel]

Is L-Smash deliberately snubbing XP these days? The latest version (r784) won't run on XP. Neither would r783.

have you tried L-SMASH-Works-r783-20150223-32bit-XP.7z (https://www.dropbox.com/sh/3i81ttxf028m1eh/AAABkQn4Y5w1k-toVhYLasmwa?dl=0)?

[hello_hello]

have you tried L-SMASH-Works-r783-20150223-32bit-XP.7z (https://www.dropbox.com/sh/3i81ttxf028m1eh/AAABkQn4Y5w1k-toVhYLasmwa?dl=0)?

No, but I will.
Thanks.

[the_weirdo]

Is L-Smash deliberately snubbing XP these days? The latest version (r784) won't run on XP. Neither would r783.

I was busy last time so I didn't answer this. L-SMASH Works itself isn't "deliberately snubbing" XP, as it doesn't require features which are exclusive on later Windows versions. It's just me who don't support XP in my builds anymore (since the begining of 2015). As I'm only sharing _my_ builds to anyone who needs them, I'm not responsible for providing builds that are as compatible as possible.

L-SMASH Works and its dependencies are open-source, the toolchain is also free, anyone can grab the source and build it themselve. Someone may provide a XP-compatible version. But it's not me. Sorry about that.

[StainlessS]

It's just me who don't support XP in my builds anymore

Well at least please let people know of the limitations everywhere that you post your links.
Thank you.

[TheFluff]

I have a modest proposal: how about you upgrade to an OS from the last decade

[burfadel]

Yes, I shouldn't need to post this in another thread, and yet I am :).

Windows XP.

Compiling for Windows XP and making it universal is detrimental to users of Windows 8.1 etc.