L-SMASH Source

[jpsdr]

Ok, i've figure out the issue.
Like a lot of people, VS2015 doesn't like when things are too tight, it needs space, so writing this :

"Failed to find %s track %" PRIu32 ".\n"

instead of this :

"Failed to find %s track %"PRIu32".\n"

solves the issue.
Is there someone i can provide/send a diff patch by mail to put on the github ?
Or should i try to make a PR ?

[WorBry]

It's L-SMASH itself that would need to be built shared, not L-SMASH-Works. The problem is that L-SMASH only uses -fPIC when building as a shared library. You have basically two options here: rebuild it as shared like TheFluff suggested, or rebuild as static but force the use of -fPIC in the CFLAGS (by appending ./configure with --extra-cflags="-fPIC"). checkinstall should be smart enough to do the general update tango so that you don't have a spare liblsmash.a without -fPIC laying around, but if you'd rather uninstall first, the command would be sudo apt-get purge lsmash.

EDIT: Beat to it. Oh well.

EDIT 2: Also, if you rebuild L-SMASH as shared, the checkinstall command would need the --stripso=yes --addso=yes options added to it.

Thanks all,

And the good news is I'm now up and running, on Kubuntu 15.10 (AMD 64) at least.

No problem opening native HD AVC (mp4 and mov) clips from various camcorders/DLSR's. But I cannot open AVCHD.mts clips from Canon, Panasonic and Sony models at all. Which is a bit frustrating, as it was my main reason for wanting to try L-Smash.

Loading a basic VapourSynth script into VSEditor:

import vapoursynth as vs
core = vs.get_core()
clip = core.lsmas.LWLibavSource(source=r"Path..../TestHFG10.mts",fpsnum=30000,fpsden=1001)
clip.set_output()

Opening Preview, there is brief pause and then VSEditor crashes. An lwi index file is generated though.

Searching for answers I came across this case:

https://github.com/VFR-maniac/L-SMASH-Works/issues/42

Ironically, the downloaded "sample.m2ts" clip from that post opened just fine with LWLibavSource. Tried running a gdb test (as cited in that thread) with my own mts clips:

Example gdb vsedit:
(note, the script file was named LSmashSource.vpy, but it was using LWLibavSource as above)

~$ gdb --args vsedit /home/bryan/Documents/LSmashSource.vpy
GNU gdb (Ubuntu 7.10-1ubuntu2) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from vsedit...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/bin/vsedit /home/bryan/Documents/LSmashSource.vpy
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffeba4d700 (LWP 2099)]
QWidget::insertAction: Attempt to insert null action
[New Thread 0x7fffda18d700 (LWP 2100)]
[Thread 0x7fffeba4d700 (LWP 2099) exited]
[Thread 0x7fffda18d700 (LWP 2100) exited]
[Inferior 1 (process 2095) exited normally]
(gdb) quit


VSEditor opened with "Failed to open the file........" error message.

Not sure where to go next with this. I can post a sample lwi index file if it would give any clues.

No problem opening these AVCHD mts files with LWLibavSource in AVISynth, by the way.

[qyot27]

gdb wouldn't be much use to you the way you've built the libs; it requires leaving all the debug symbols in the binaries (all of them, preferably; FFmpeg, L-SMASH, L-SMASH-Works, VapourSynth, and VSEditor).

If you have the hard drive space, I'd just run the *.mts files through mkvmerge first and only work with the Matroska copies.

[WorBry]

gdb wouldn't be much use to you the way you've built the libs; it requires leaving all the debug symbols in the binaries (all of them, preferably; FFmpeg, L-SMASH, L-SMASH-Works, VapourSynth, and VSEditor).

Ah, OK.

If you have the hard drive space, I'd just run the *.mts files through mkvmerge first and only work with the Matroska copies.

Already tried that. The remuxed mkv files behave the same way as the mts files.

[speedyrazor]

Hi, I have hit a problem opening a Quicktime Prores in Avisynth, which has 1 audio track with 10 channels of audio in it. It works with 1 track with 2 audio channels, but with this file that has 10 channels I am getting this error:

LSMASHAudioSource: failed to open resampler.
(C:\Users\me\Desktop\test.avs, line1)

Here is the avs script:

LSMASHAudioSource(source="10_ch_test.mov", track=2)

I am using 'LSMASHSource-AviSynth-plugin-r859-static-32bit'.
I will attached a small movie sample later. Is there anything i can do to get round this, is it a channel limitation?

Kind regards.

Attached is a small 1 frame sample, which shows the error.

[WorBry]

Interesting.

It's not as if I've burned my bridges with Windows and AVISynth though, and I have other reliable options there for AVCHD (DGDecIM) - so it's not such a big deal really.

I just find it a bit odd that the AVISynth version of L-Smash I have installed (r804 in MeGUI) has no problem opening these mts files. And also why this VapourSynth version should open Sony AVHD m2t clips without any issues. So whatever it is that causing this version to choke on (Canon and Panasonic) mts files specifically must surely be pretty minor? Like I said, an index file is generated.

[kuchikirukia]

http://s000.tinyupload.com/?file_id=72586136885519263005

DGDecode (CPU=4, info=3) + QTGMC: Works well enough to srestore
LSMASH + QTCMC: Horribly broken
FFMS2 + QTGMC: Horribly broken

Why the vast difference?

[Sharc]

ffms2 (still) has issues with interlaced sources afaik.
Don't know about LSMASH though.

[LigH]

Too few details. Before I download it: What do you mean by "broken"? How does it look without QTGMC? Did you enforce the correct field order (not all source plugins tell them correctly in all cases)?

[kuchikirukia]

Too few details. Before I download it: What do you mean by "broken"? How does it look without QTGMC? Did you enforce the correct field order (not all source plugins tell them correctly in all cases)?

It seems to switch to BFF. DGDecode handles this.

[WorBry]

It seems to switch to BFF...

I've just run a few tests with a couple of 1080/50i and 1080/60i AVCHD.mts clips from a Panasonic DSLR and Canon camcorder. I cant test them in Vapoursynth with LWLibavSource because they won't load at all (as reported above). But testing with ffms2 in Vapoursynth the output was definitely BFF. That's on the evidence of the back and to movement of the fields with:

clip = core.std.SeparateFields(clip, tff=True)

...and likewise of the frames after:

clip = haf.QTGMC(clip, Preset='Medium', TFF=True)


Seen both on Preview in VSEditor and after encoding to UTVideo.

Tried adding:

clip = core.std.SetFrameProp(clip, prop="_FieldBased", intval=2)

after ffms2 and before QTGMC, but it made no difference.

Testing the same clips with ffms2 and LWLibavSource in AVISynth, the QTGMC 50/60p encodes (UTVideo YV12) looked OK. But when doing the time honoured field order test:

SeparateFields()
AssumeTFF()

and looking at the movement of the fields in VirtualDub. With ffms2, I noticed that upon seeking to any particular frame, the movement was at "first back and to", suggesting BFF, but then after advancing a number of frames (I didn't count) it reverted back to a TFF pattern. But if I just let the script run through from frame 0 to finish, it looked to be all TFF. Testing with LWLibavSource the field movement pattern was consistently TFF when seeking to any frame.

That's all I can say.

[Sharc]

I did few tests again with ffms2. Forget it as a source filter for interlaced material. It messes up with the field order.
LWLibavVideoSource seems to work. I prefer DG tools though (DGdecNV).

[WorBry]

LWLibavVideoSource seems to work

But not the Vapoursynth version with AVCHD.mts files:


..No problem opening native HD AVC (mp4 and mov) clips from various camcorders/DLSR's. But I cannot open AVCHD.mts clips from Canon, Panasonic and Sony models at all. Which is a bit frustrating, as it was my main reason for wanting to try L-Smash.

Loading a basic VapourSynth script into VSEditor:

import vapoursynth as vs
core = vs.get_core()
clip = core.lsmas.LWLibavSource(source=r"Path..../TestHFG10.mts",fpsnum=30000,fpsden=1001)
clip.set_output()

Opening Preview, there is brief pause and then VSEditor crashes. An lwi index file is generated though.

Searching for answers I came across this case:

https://github.com/VFR-maniac/L-SMASH-Works/issues/42

Ironically, the downloaded "sample.m2ts" clip from that post opened just fine with LWLibavSource......

......Not sure where to go next with this. I can post a sample lwi index file if it would give any clues.

No problem opening these AVCHD mts files with LWLibavSource in AVISynth, by the way.

Is there no interest in fixing this? Despite the DSLR video boom and increasing trend to native progressive HD-AVC, and now UHD-AVC, mov and mp4 formats, there are still a lot of people recording AVCHD.

[TheFluff]

you can't randomly swap ffmpeg dll's around, the abi changes all the damn time, of course it's not gonna work

just link it statically like everyone else; plugins depending on shared libraries is an awful idea

[lansing]

Hi, I wanted to save the cache index file outside the source file location so that I can directly read those huge blu-ray video without extracting to local. I am looking for something like saving them to the same directory as the script file instead.

I had read the document for the cache argument but I couldn't find an option to do it.

[Music Fan]

Hi, I wanted to save the cache index file outside the source file location so that I can directly read those huge blu-ray video without extracting to local. I am looking for something like saving them to the same directory as the script file instead.

I had read the document for the cache argument but I couldn't find an option to do it.
You can try cache=false, if I understand well what you need (it creates index outside the RAM), but I believe you can't select directory to write this index.

[dipje]

For the record, I've stopped using FFMS2 (in Vapoursynth) because of issues with 10-bit ProRes and my Sony DSLR AVCHD files (1080p25 / 1080p50).

That's why I switched to l-smash. At the moment, that was the thing that's working.

And it's still working fine on r875 (x64 Vapoursynth that is). So not all AVCHD streams are crashing apparently, mine are still working fine.

[TalasNetrag]

When trying to open a H.264 10bit video, LWLibavVideoSource only return a greenish artefact picture.

[stax76]

It's over a year the author posted here: http://forum.doom9.org/search.php?searchid=7538802

He posted 5 days ago at the issue tracker: https://github.com/VFR-maniac/L-SMASH-Works/issues

For a bug report it's good to have some script, log file and a sample of course.

[sneaker_ger]

When trying to open a H.264 10bit video, LWLibavVideoSource only return a greenish artefact picture.
That's normal. AviSynth only natively supports 8 bit so l-smash's 10/16 bit output looks psychedelic. The simplest way to handle the problem is to tell l-smash to dither down to 8 bit:
LWLibavVideoSource("source.mkv", format="YUV420P8")

http://forum.doom9.org/showthread.php?t=173088
http://avisynth.nl/index.php/High_bit-depth_Support_with_Avisynth

[TalasNetrag]

That's normal. AviSynth only natively supports 8 bit so l-smash's 10/16 bit output looks psychedelic. The simplest way to handle the problem is to tell l-smash to dither down to 8 bit:
LWLibavVideoSource("source.mkv", format="YUV420P8")]

FFVideoSource opens the video without any extra parameters, does that mean that FFVideoSource converts it down by itself? Is there any downside going from 10->8->10 (or 12)bits?

[sneaker_ger]

does that mean that FFVideoSource converts it down by itself?
Yes, FFVideoSource dithers everything to 8 bit without asking.

Is there any downside going from 10->8->10 (or 12)bits?
Well, you lose a bit of intermediate precision which means you might introduce some extra banding. Is it bad enough to worry about? Maybe, maybe not. It's subjective.

[Groucho2004]

Is there any downside going from 10->8->10 (or 12)bits?
You don't have to. Have a look at Dithertools (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

[TalasNetrag]

You don't have to. Have a look at Dithertools (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

Ok, after looking at Dithertools I have this setup now:
1. Open h.264 10bit video with LWLibavVideoSource(stacked=true)
2. Do nothing
3. Dither down to 10bit with Dither_quantize()
4. Reencode the video with x264-10bit

The Script:

LWLibavVideoSource("test.mkv", format="YUV420P16", stacked=true)
#~ AssumeFPS("ntsc_video")
Trim(20511, 22248)
Dither_quantize(bitdepth=10, reducerange=true)
Dither_out()


Encoding:

avs2yuv -raw "test.avs" -o - | x264-10bit --preset slower --tune animation --crf 18 --demuxer raw --input-depth 10 --input-res 768x576 --fps 29.97 --output "test.h264" -


Is this how it's supposed to be done? Because thats kinda inconvenient just to avoid dithering down to 8bit.

Use Bitdepth(from=16,to=8) from my HDRCore tools
I want to avoid going down to 8bits.

[sneaker_ger]

If you don't want to do any further processing (except Trim()), just re-encode, you don't need any extra plugins:
LWLibavVideoSource("test.mkv")
Trim(20511, 22248)
avs2yuv -raw "test.avs" -o - | x264-10bit --preset slower --tune animation --crf 18 --demuxer raw --input-depth 10 --input-res 768x576 --fps 30000/1001 --output "test.h264" -

[Music Fan]

3. Dither down to 10bit with Dither_quantize()
Why do you have to dither down to 10 bit while your source is already in 10 bit ?

[LigH]

Because thats kinda inconvenient just to avoid dithering down to 8bit.


I want to avoid going down to 8bits.

But AviSynth only supports 8 bits per color component.

If you want a frameserver which supports more than that natively, without applying workarounds, try something different than AviSynth. Maybe VapourSynth.

[qyot27]

Is this how it's supposed to be done? Because thats kinda inconvenient just to avoid dithering down to 8bit.
Just use a 10-bit x264 that includes the AviSynth 16-bit patch (https://github.com/astrataro/x264_tMod/commit/42227cae1291165f1a39ee5536b1646e75534540) so you can stitch interleaved >8bit back together. No piping required. Any other way to do this while using AviSynth requires piping.

For that matter, that build of x264 might very well have that patch in it, in which case the pipe would be absolutely unnecessary except for the part where you're using the wrong >8bit hack format.

[TalasNetrag]

If you don't want to do any further processing (except Trim()), just re-encode, you don't need any extra plugins

For now I only wanted to see how to open and encode the video, further processing is supposed to be inserted into the "2. Do nothing" part later.

Why do you have to dither down to 10 bit while your source is already in 10 bit ?

Because the documentation of Dithertools said that I shoud dither it down from stack16 to 10bit for the encoder, or use --input-depth 16 if not dithering down and let the encoder to it.

[LigH]

further processing is supposed to be inserted into the "2. Do nothing" part later.

This "further processing" would have to be aware of such 16-bit-per-component workarounds. You shall not expect every filter to know how to handle such special videos.

[TalasNetrag]

Using L-SMASH 32bit with Vapoursynth portable 32bit always crashes regardles of input file. Using Vapoursynth 64bit and L-SMASH 64bit works fine.

[kuchikirukia]

Problem file:
http://www.filedropper.com/title001-002

LSMASH thinks there's 86210 fields in this episode, dgdecode 86602. QTGMC can put it together fed by dgdecode, but with LSMASH it's not just wrong field order, it comes out combed. QTGMC normally nukes combing from orbit.

http://s15.postimg.org/xcf7tavtz/dgdecode_qtgmc.png (http://postimg.org/image/xcf7tavtz/)

http://s15.postimg.org/wbez46euf/lsmash_qtgmc.png (http://postimg.org/image/wbez46euf/)

[poisondeathray]

Problem file:
http://www.filedropper.com/title001-002

LSMASH thinks there's 86210 fields in this episode, dgdecode 86602. QTGMC can put it together fed by dgdecode, but with LSMASH it's not just wrong field order, it comes out combed. QTGMC normally nukes combing from orbit.


I get identical results for your sample in dgdecode and l-smash r859 . Which version of l-smash are you using ?

l-smash/ffms2 are less reliable when leaving mpeg2 in mkv, and will sometimes report the wrong framerate. But the field/frame count was the same, just the fps "off" when left in mkv (which you could fix with assumefps(30000,1001) )

[kuchikirukia]

No idea what version since it doesn't say. 13,036,032 bytes created ‎October ‎29, ‎2015, ‏‎10:35:26 PM.

It did it from m2v too.
I'll update.

Oh, MeGUI's updater says r804. Since I don't recall manually updating it's probably still that.

e: I'm still getting the same issue with r877, and from m2v. I remade the lwi's.

[ravewulf]

RGB24 support partly broken:
Reading an RGB24 clip (created with AviSynth+ffmpeg, tested ffvhuff and ffv1 codecs) results in error "LWLibavVideoSource: bgr0 is not supported." Manually setting LWLibavVideoSource to use format="rgb24" works. So my assumption is that it isn't recognizing that bgr0 = RGB24. RGB32/RGBA works as expected.

Issue has been submitted at https://github.com/VFR-maniac/L-SMASH-Works/issues/51

[LigH]

@ qyot27:

In January, you discussed with an3k (http://forum.doom9.org/showthread.php?p=1753032#post1753032) how to build L-SMASH Works under Linux. Unfortunately, this procedure seems to require a few packages a prepared MSYS environment (like the one provided by XhmikosR) does not contain: checkinstall, apt, git, sudo (well, sudo may not be necessary at all due to the lack of a secure user management in MSYS; I made git available by installing Git-for-Windows). So I wonder how to make an LSMASHSource.dll in MSYS/MinGW-x64 if that is possible. Will that lead to previously compiling ffmpeg (enabling shared, avresample, pic) too?
_

Well, probably, to include libavformats / libavcodec.

In this case I guess that the media-autobuild_suite may be a good base to continue on, it can build ffmpeg shared...

[qyot27]

@ qyot27:

In January, you discussed with an3k (http://forum.doom9.org/showthread.php?p=1753032#post1753032) how to build L-SMASH Works under Linux. Unfortunately, this procedure seems to require a few packages a prepared MSYS environment (like the one provided by XhmikosR) does not contain: checkinstall, apt, git, sudo (well, sudo may not be necessary at all due to the lack of a secure user management in MSYS; I made git available by installing Git-for-Windows). So I wonder how to make an LSMASHSource.dll in MSYS/MinGW-x64 if that is possible. Will that lead to previously compiling ffmpeg (enabling shared, avresample, pic) too?
_

Well, probably, to include libavformats / libavcodec.

In this case I guess that the media-autobuild_suite may be a good base to continue on, it can build ffmpeg shared...
I don't recall ever trying to build L-SMASH-Works under MSys (well, MSys2, since it has a proper - and up-to-date - package management system), but as I understand it the issues with the libs are slightly different on Windows in regards to PIC and whatnot.

The VapourSynth plugin should be able to be compiled fully static, save for LSMASHSource.dll in the final step; it's the AviSynth plugin that can't be built by MinGW-w64 and would seem to enforce the shared libav* libs thing (but I haven't tried building the AviSynth plugin for a long time, since I hate dealing with Visual Studio). g++ can't build AviSynth plugins because the C++ name mangling schemes and various other errata are incompatible, so the only way to get MinGW-w64 to build a working LSMASHSource AviSynth plugin would be to implement a C-plugin interface for it.

apt is simply the package management system on Debian-family Linux distros, and checkinstall allows users to build packages quickly so they're managed through apt rather than the mess that can ensue by having non-logged 'make install' steps littering /usr or /usr/local. With a regular 'make install', the only way to deal with later uninstallation is to run 'make uninstall' from the source tree, which means either keeping the source tree around (taking up space for something you're probably not touching), or re-running the build process for the sole reason of uninstalling the application/library (which is a waste of time).

For MSys this is moot since package management is virtually nonexistent there, and with MSys2 you might be able to manage something by playing around with pacman and PKGBUILD recipes the same way it'd be done on Arch Linux (but I have no experience with that; I use MSys2 as a means of having the GNU userland and related utilities available on Windows and keeping them up-to-date, not for compiling things).

The recent announcement that Windows 10 will have an Ubuntu-based Linux compatibility layer (https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux) will probably allow for Windows users to follow the exact same steps as the Linux cross-compiling tutorials, provided they set up the MinGW-w64 environment first (again, covered by said cross-compiling tutorials). Heck, as long as the binaries aren't kernel-level, the cross-compiling probably wouldn't even be necessary.

[LigH]

MSYS2 is the base of jb-alvarado's media-autobuild_suite, so I guess something could be done inside here. I know they use package management (e.g. pacman) to keep it up-to-date, and they use several multi-developer source managers (at least svn, git, and hg). So I guess I could ask there if adding AVS/VPY Source DLLs could be achieved easily. The rest of the required packages is already downloaded and built anyway.

[jones1913]

So I guess I could ask there if adding AVS/VPY Source DLLs could be achieved easily.
For vpy plugins that should be possible. But like qyot said, avs plugins can not be build with gcc, only with visual studio.

[Groucho2004]

But like qyot said, avs plugins can not be build with gcc, only with visual studio.
Sorry to be pedantic but this should read "avs cpp plugins can not be build with gcc, only with visual studio." C-plugins can be built with gcc.

[jones1913]

Ah right. The majority of avs plugins are cpp plugins though (afaik).

The only 'c' plugin I know of is the 'ffms2 c' plugin, but I never used it.

[stax76]

Ah right. The majority of avs plugins are cpp plugins though (afaik).

The only 'c' plugin I know of is the 'ffms2 c' plugin, but I never used it.

ffms2 c version leaks memory unlike cpp version, I think there is a yadif c version, yadifmod2 is cpp

[Groucho2004]

Ah right. The majority of avs plugins are cpp plugins though (afaik).

The only 'c' plugin I know of is the 'ffms2 c' plugin, but I never used it.
Some C-plugs, although most of them quite dated:
avscurveflow.dll
avsshock.dll
equlines.dll
ibob.dll
smartdecimate.dll
transition.dll
avsinpaint.dll
yadif.dll

[StainlessS]

Of the C Plugins mentioned by G2004 above, only the last two are v2.5 C plugins, the rest are version v2.0 plugs (at least the ones that I have).

ie v2.5

avsinpaint.dll
yadif.dll

[Motenai Yoda]

Ok, after looking at Dithertools I have this setup now:
1. Open h.264 10bit video with LWLibavVideoSource(stacked=true)
2. Do nothing
3. Dither down to 10bit with Dither_quantize()
4. Reencode the video with x264-10bit

Is this how it's supposed to be done? Because thats kinda inconvenient just to avoid dithering down to 8bit.

I want to avoid going down to 8bits.
Pretty simple

- long way

1. Open h.264 10bit video with LWLibavVideoSource(stacked=false)
2. Reencode the video with x264-10bit
using
x264-10bit --preset slower --tune animation --crf 18 --input-depth 16 --output "test.h264" "test.avs"
even if I suggest you to pipe with avs2yuv or avs4x26x to an x64 x264's build.

- short way

1. Reencode the video with x264-10bit
using
x264-10bit --preset slower --tune animation --crf 18 --output "test.h264" "test.mkv"

[sneaker_ger]

I think x264 ignores --input-depth when using AviSynth input so your first example won't work.

[Motenai Yoda]

Yep for raw input only XD but I think it should be extended to avs input too;
also is more wrong coz lsmash return a 10bit reduced clip not a 16bit ones @_@.

[qyot27]

Not even a month ago. (http://forum.doom9.org/showpost.php?p=1760495&postcount=542) The patch to use --input-depth with AviSynth has been around a loooong time.

[Motenai Yoda]

Not even a month ago. (http://forum.doom9.org/showpost.php?p=1760495&postcount=542) The patch to use --input-depth with AviSynth has been around a loooong time.

well Komisar don't apply this patch to his own builds, and taro's last builds is 2yo

[Reel.Deel]

well Komisar don't apply this patch to his own builds, and taro's last builds is 2yo

Try jpsdr's t_mod built: https://github.com/jpsdr/x264/releases