L-SMASH Source

[videoh]

We're having so much fun I decided to try with GPU encoding for this use case we are discussing:

DGSource() + NVENCC + 2080 Ti + 7700K: 1:05.

I tried to choose settings that produced results perceptually approximating x264 medium.

[LigH]

Isn't the video decoding done in a separate PureVideo decoder chip which may not have developed as much as the 3D rendering and shader operations? ... The closet was the room with the least needs for space in this house.

[FranceBB]

That means the video renderer can't keep up because of the limited PCIe bandwidth? Very interesting.

Also the number of lanes that the CPU has with the GPU is important. For instance, Xeon CPUs like the W-3265M has 64 PCI-E lanes, while its consumer counterpart probably has less lanes. The thing is that it's not only a matter of GPU, but also a matter of which CPU you are using. As a matter of fact, having a very good GPU with a not so good CPU will limit its efficiency.

[redbtn]

Can I safely index MKV (hevc) with audio tracks and subs for now? I read somewhere earlier that i need remux MKV with only video track for better indexing, cuz another tracks can impact on accuracy.

[videoh]

Isn't the video decoding done in a separate PureVideo decoder chip which may not have developed as much as the 3D rendering and shader operations? ... The closet was the room with the least needs for space in this house. Quite right, LigH. Seems to me that the main nVidia development effort goes into gaming-relevant things. We really need super high PCI bandwidths too.

[MeteorRain]

Can I safely index MKV (hevc) with audio tracks and subs for now? I read somewhere earlier that i need remux MKV with only video track for better indexing, cuz another tracks can impact on accuracy.

Fairly safe. It used to index all other tracks and would create huge index file. Now they are all skipped. Let us know if you hit issues again.

[StvG]

@HolyWu, I built LSMASHSource from your repo using this CMake file (https://pastebin.com/raw/cMFdyuye) taken from @MeteorRain repo. When tested for seeking issues with that file (https://www.mediafire.com/file/dldzx9smmbxf60m/vc1_sample.mkv/file) my build has errors but your build is fine. Something wrong with my building process? ffmpeg 4.2.1 / 4.3 used without difference.
There is patch for replacing avresample with swresample in your repo but avisynth exlibs - #pragma comment( lib, "libavresample.a" ).

[StvG]

Thanks for the info.

[FNSCAR]

@HolyWu ,I downloaded your build from release page of your github but there is no vsLSMASHSource.dll in folder.
Does it support vapoursynth?

[ChaosKing]

@HolyWu ,I downloaded your build from release page of your github but there is no vsLSMASHSource.dll in folder.
Does it support vapoursynth?

It works for avisynth and vapoursynth.

[FNSCAR]

It works for avisynth and vapoursynth.

I update vapouraynth to latest R47.2 and it works.Thank you.

[DJATOM]

R47.2 is buggy. Stick with R46 or go to latest R48 test. It contains fixes for all spotted bugs of R47.

[Atak_Snajpera]

Question: Would be possible to use multiple threads to speed up indexing process? For example by using some kind of chunk indexing?
I'm asking because NVMe SSDs with read speed of 3GiB/s+ are getting much more affordable. I have Xeon E5-2690@3.2GHz (8C/16T) and I see that i'm experiencing a bottleneck due to single threaded code.
In this example I'm indexing 4k movie (~50Mbps) from RAM disk...
https://i.imgsafe.org/c7/c7b40e940e.png
1,24GiB/s is a max I can get on single core. ffmsindex is even more cpu intensive (only ~0,5GiB/s).

Currently indexing of ~60 GiB 4k movie takes ~1 minute. With proper multi-threading we could easily reduce indexing time to ~15s!

[kedautinh12]

https://github.com/HolyWu/L-SMASH-Works/releases/latest

Uploaded new binary which ditches libaom and is built by clang-cl. The binary size is smaller than the previous ICL build and decoding speed seems to be marginally faster.




No problem. Here (https://drive.google.com/open?id=1Wja3h-8UmgzzZ7ID2cacoRsoKAmGnsmi).

Your Latest still error with .vp9, add this video and check
https://drive.google.com/file/d/1FAl-hS_jsvUGaHFhpK8sImwc91PUt2xK/view?usp=drivesdk

[stax76]

Question: Would be possible to use multiple threads to speed up indexing process? For example by using some kind of chunk indexing?
I'm asking because NVMe SSDs with read speed of 3GiB/s+ are getting much more affordable. I have Xeon E5-2690@3.2GHz (8C/16T) and I see that i'm experiencing a bottleneck due to single threaded code.
In this example I'm indexing 4k movie (~50Mbps) from RAM disk...
https://i.imgsafe.org/c7/c7b40e940e.png
1,24GiB/s is a max I can get on single core. ffmsindex is even more cpu intensive (only ~0,5GiB/s).

Currently indexing of ~60 GiB 4k movie takes ~1 minute. With proper multi-threading we could easily reduce indexing time to ~15s!

Maybe you are asking the wrong question, how about MakeMKV creating an index? Answer: It's just code, anything is possible.

[kedautinh12]

https://down.7086.in/AviSynthPlus%20Filters/LSMASHSource-r935%2B34.zip

Stop indexing progress spamming

-- Now only refresh at every 1%.

Still error with vp9, only old ver r929 of VFR-maniac still fine

[poisondeathray]

Your Latest still error with .vp9, add this video and check
https://drive.google.com/file/d/1FAl-hS_jsvUGaHFhpK8sImwc91PUt2xK/view?usp=drivesdk

This is AVC. Works ok. Rename the file characters

[kedautinh12]

This is AVC. Works ok. Rename the file characters

I was renamed and load ok but still error with image in video

[poisondeathray]

I was renamed and load ok but still error with image in video

what error? what time ?

[kedautinh12]

what error? what time ?

When i load video with LWLibavVideoSource still error with L-SMASH-Works_20190917 and LSMASHSource-r935+34
https://drive.google.com/open?id=115BKlC1q0kjdEjgsmqYi9jrKPfOO2smu

But i load ok with L-SMASH-Works r929
https://drive.google.com/open?id=16ckCimru9cIALC6wH_G738RIUaCO2MtC

[poisondeathray]

I see no errors with 20190917

Are you sure it's the correct video ? Earlier you said ".vp9" . This is AVC in MP4.

[kedautinh12]

I see no errors with 20190917

Are you sure it's the correct video ? Earlier you said ".vp9" . This is AVC in MP4.

Ah sorry my fault, this video right
https://drive.google.com/open?id=1y0MdC98p4AGr52nddZwXqzCMQwNSrTlx
Error frame with vp9 in .MKV

[Atak_Snajpera]

Creation of audio index file needs some optimalization because file size can be extremely large.
Example: 10 min TrueHD
https://i.postimg.cc/BQLdyWsH/Capture.png

It looks like that every 1 minute of TrueHD audio adds extra 10 MiB to index file. With typical 100+ minute movie index file will be over 1 GiB! Parsing that big ass file on single core is just super ineficient.

Question: Do we really need those repeated information in each line? From what I see only values for POS,PTS,DTS are changing. Furthermore values PTS and DTS are increasing in predictable manner.

<LSMASHWorksIndexVersion=0.0.2.0>
<LibavReaderIndexFile=15>
<FileSize=526166830>
<FileHash=0x7b6795d7>
<LibavReaderIndex=0x00000180,1,truehd>
<ActiveVideoStreamIndex>-0000000001</ActiveVideoStreamIndex>
<ActiveAudioStreamIndex>+0000000000</ActiveAudioStreamIndex>
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=0,PTS=0,DTS=0,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=576,PTS=75,DTS=75,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=740,PTS=150,DTS=150,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=902,PTS=225,DTS=225,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1050,PTS=300,DTS=300,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1194,PTS=375,DTS=375,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1340,PTS=450,DTS=450,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1476,PTS=525,DTS=525,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1624,PTS=600,DTS=600,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1762,PTS=675,DTS=675,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=1912,PTS=750,DTS=750,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=2058,PTS=825,DTS=825,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=2196,PTS=900,DTS=900,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=2346,PTS=975,DTS=975,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40
Index=0,Type=1,Codec=86060,TimeBase=1/90000,POS=2490,PTS=1050,DTS=1050,EDI=0
Channels=8:0x63f,Rate=48000,Format=s32,BPS=24,Length=40

[Atak_Snajpera]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191115/L-SMASH-Works_20191115.7z


Update to FFmpeg-20191114-73ee53f.
VapourSynth: Export mastering display metadata and content light level in frame properties.
VideoSource: Improve capability check in prefer_hw.
LWLibav: Fix VP9 decoding issue with superframes.
VideoSource: Enable AV1 decoding via libdav1d.
Add parameter ff_loglevel.
LWLibav: Adjust the structure of index file to reduce file size.


Thank you for your excellent work!

[fg118942]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191115/L-SMASH-Works_20191115.7z


Update to FFmpeg-20191114-73ee53f.
VapourSynth: Export mastering display metadata and content light level in frame properties.
VideoSource: Improve capability check in prefer_hw.
LWLibav: Fix VP9 decoding issue with superframes.
VideoSource: Enable AV1 decoding via libdav1d.
Add parameter ff_loglevel.
LWLibav: Adjust the structure of index file to reduce file size.


HolyWu, thank you so much for fixing the issue of VP9.
However, using LSMASHVideoSource in version 20191115 will cause Access Violation.

[2ndR]

Perfect

[hydra3333]

Thank you HolyWu.
I have just visited this thread for the first time in a while.
I have changed my icky x264 build process to use your LSW git accordingly :)
Nice work !!!!


edit:
PS is there any possibility you may turn on the Issues tab in your github fork ?
Apparently,
- Go to the Settings page of your fork.
- Check the box next to Issues
- You can now file issues on your own fork and they will not be placed in the main repo.

[FranceBB]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191116/L-SMASH-Works_20191116.7z


LibavSMASH: Fix access violation.


Thank you, once again! :)

[Patman]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191116/L-SMASH-Works_20191116.7z


LibavSMASH: Fix access violation.


Hi HolyWu,

thanks for your build. I think there is a problem when some special characters like ' are included in the file name. Can you please check that? For me the LibavSMASHSource Filter is corrupted. LWLibavSource works.

[StainlessS]

Is the offending character perhaps slanting single quote (`) ?

EDIT: Normal Single quote ('), Grave (`), [B]Acute [forwards slant] (ˊ).

EDIT: Patman, post your actual problem filename.

EDIT: I note these new functions in Avs+:- http://avisynth.nl/index.php/Internal_functions#String_functions

StrToUtf8

StrToUtf8(string) AVS+
Converts string from ANSI to UTF8.

StrFromUtf8

StrFromUtf8(string) AVS+
Converts string from UTF8 to ANSI

[LigH]

Not to forget typographic single quotes in nationally specific styles...

[Patman]

Cannot reproduce.

AVS:
https://i.postimg.cc/LqkvWJYd/test.png (https://postimg.cc/LqkvWJYd)

VS:
https://i.postimg.cc/xJwCb15C/test.png (https://postimg.cc/xJwCb15C)

Thanks for your fast reply. The failure was in call of the file. Thanks again.

EDIT: Patman, post your actual problem filename.

The Redwood's - single quote

[StainlessS]

The Redwood's - single quote
So I'm assuming that its "The Redwood's" with ordinary single quote & without a path, and also without relative path ie not ".\The Redwood's".
SPACE in filename can also be a problem for some tools, HASH(#) used to be problem in MeGUI input filename,
used to interpret it as part of some kind of command, no longer a problem for quite some time.
Assuming that problem exists without using eg MeGUI, and only load into Avsinth (or avs+).
Does it work OK when single quote stripped from filename (and of course the call to src filter).

EDIT: Some commands/calls to functions/filters fail on W7+ where used to work on XP (dont know about Vista),
M$ did something that no longer converts eg "name.xxx" to a proper relative path, used to work fine on XP.

EDIT: IIRC, W7+ can interpret filename without path/relative path, to "C:\"+Filename, which then re-targets to Virtualstore (or something like that).

[Patman]

So I'm assuming that its "The Redwood's" with ordinary single quote & without a path, and also without relative path ie not ".\The Redwood's".
SPACE in filename can also be a problem for some tools, HASH(#) used to be problem in MeGUI input filename,
used to interpret it as part of some kind of command, no longer a problem for quite some time.
Assuming that problem exists without using eg MeGUI, and only load into Avsinth (or avs+).
Does it work OK when single quote stripped from filename (and of course the call to src filter).

EDIT: Some commands/calls to functions/filters fail on W7+ where used to work on XP (dont know about Vista),
M$ did something that no longer converts eg "name.xxx" to a proper relative path, used to work fine on XP.

Hi StainlessS,

there was a failure in the code structure of my command. I used vapoursynth to test some plugins and functions. The full path is E:\x264\test\The Redwood's.mp4. After i corrected the code in the command line, everything works fine. Thanks for the help.

[Rumbah]

I noticed that using the cache file option with version 20191116 and Vapoursynth 64bit results in two lwi files being created.

Using the code line below results in the file creation at Q:\%folder%\video.lwi and the standard file at source + ".lwi" as well.
core.lsmas.LWLibavSource(source=r"%Source%", cachefile=r"Q:\%folder%\video.lwi")
Or do I use the option wrong?

[StainlessS]

Cannot reproduce. :confused::confused::confused:

I'm guessin' that Rumbah did not delete existing files beforehand [just a silly faux pas - suggest Rumbah retries after delete both].

[Rumbah]

I just tested again with fresh files an folders and got the same result. Will try to get a reproducible test case to share.

[kedautinh12]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191116/L-SMASH-Works_20191116.7z


LibavSMASH: Fix access violation.


Your build very good with many .ts but with some .ts duration of video doesn't match with duration of audio
Example:
https://drive.google.com/a/my.smccd.edu/file/d/1rUauSk-C-QxYxxJDgkFGZDV5CiiGomr1/view?usp=drivesdk

[Rumbah]

Well, sorry for the false alarm, it works now, I had an mistake in my program.

[StainlessS]

Rumbah, can you enlighten us on what the mistake was, might give a clue for some other problem in future [also we like a good giggle :) ].

[Rumbah]

I have a script that converts a video with x264 and splits the source to X parts to encode the parts in parallel.
I found the performance to be better with less threads and more parts than with straight 36 threads per encode (a Ryzen 3900X with 12cores/24threads).

So I create the parts via Vapoursynth and L-Smash to get the cutting frame accurate so I have
video = core.lsmas.LWLibavSource(source=r"%Source%", cachefile=r"Q:\%directory%\video.lwi")in more than one part of the script.

First it was just video = core.lsmas.LWLibavSource(source=r"%Source%")but I wanted all the temp files in one directory for easier cleaning after finishing. And I just forgot one place to change it so the lwi file got created in both places.

[fg118942]

Your build very good with many .ts but with some .ts duration of video doesn't match with duration of audio
Example:
https://drive.google.com/a/my.smccd.edu/file/d/1rUauSk-C-QxYxxJDgkFGZDV5CiiGomr1/view?usp=drivesdk

Not patched FFmpeg, the Repeat First Field flags will not work, so you need to use the following patch.
https://gist.github.com/maki-rxrz/5a7a2c789e4369fa34853b5358fb8a29

[StainlessS]

Rumbah, thanx for the explanation.

[kedautinh12]

Not patched FFmpeg, the Repeat First Field flags will not work, so you need to use the following patch.
https://gist.github.com/maki-rxrz/5a7a2c789e4369fa34853b5358fb8a29

Oh, but how i can use this??

[fg118942]

Oh, but how i can use this??

We'll have to wait for HolyWu to upload a new version with this patch applied.
After that, write the following avs.

LWLibavVideoSource("***.ts",repeat=true)

[kedautinh12]

We'll have to wait for HolyWu to upload a new version with this patch applied.
After that, write the following avs.

LWLibavVideoSource("***.ts",repeat=true)

Nice, waiting for HolyWu

[Forteen88]

Thanks HolyWu.

[kedautinh12]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191127/L-SMASH-Works_20191127.7z


Fix MPEG-2 decoding issue with RFF flags. (maki-rxrz)
Fix interlaced H.264 decoding issue in some files.


Wow, thanks so much

[kedautinh12]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20191127/L-SMASH-Works_20191127.7z


Fix MPEG-2 decoding issue with RFF flags. (maki-rxrz)
Fix interlaced H.264 decoding issue in some files.


when i decoding LWLibavVideoSource("***.ts",repeat=true) with repeat=true can Fix MPEG-2 decoding issue with RFF flags (maki-rxrz) and Fix interlaced H.264 decoding issue in some files. Very nice

[LigH]

I found an MP4 video provided by the Mediathek of the German broadcaster ZDF. MediaInfo reports a framerate of 50 fps (720p50), AviSynth's Info() reports the same when using FFVideoSource or LwLibavVideoSource. But LSMASHVideoSource reports 0.0246 fps (1798/73073).

I wonder how to send you a sample. If I use any tool to cut out a small scene, it would create completely new headers, possibly fixing this issue. So I fear I would have to upload ~680 MB. And if you tried to download it using MediathekView, it might be geoblocked if you are outside Germany. (direct URL (https://rodlzdf-a.akamaihd.net/none/zdf/19/11/191122_sendung_hsh/4/191122_sendung_hsh_3328k_p36v14.mp4), may or may not work in a web browser)

[ChaosKing]

Well, I don't recommmend using LSMASHVideoSource as it's not frame accurate when random seeking even though it has the advantage of not requiring an index file. I once even considered dropping this filter from my fork.

Hmm I only encountered inaccurate frame seeking once with LSMASHVideoSource. I use it sometimes with big (40gb+) lossless avc+mp4 files, so it would be nice to keep it in this fork :-)

[dREV]

I wanted to try updating 32 bit MeGUI 2525's lsmash that's on the wiki page in the hopes it'll stop the "Access Violation" thing it receives after indexing several m2ts files in a row and the program accumulates high memory MB's causing it to give errors only correcting when restarting the program plus I also noticed that as I frequently use film.trim sometimes the frame will be off by 1 frame but it'll still fit the frame cut amount which is really strange but doesn't happen often thankfully.

However, I'm receiving errors after it finishes idx then goes to oneclick and stops or continues to the next job. Thought of restarting MeGUI would fix that as it does when receiving the "Access Violation" but nope.

Used L-SMASH-Works_20191127 (https://github.com/HolyWu/L-SMASH-Works/releases) and used all previous versions with nothing.

The error says this:

--[Error] [12/4/2019 5:28:47 AM] An error occurred
---[Error] [12/4/2019 5:28:47 AM] Exception message: [Fatal]: Failed to avformat_open_input.
---[Error] [12/4/2019 5:28:47 AM] Stacktrace
----[NoImage] at MeGUI.AviSynthClip..ctor(String func, String arg, AviSynthColorspace forceColorspace)
----[NoImage] at MeGUI.AvsFile..ctor(String script, Boolean parse)
----[NoImage] at MeGUI.lsmashFile..ctor(String fileName, String indexFile)
----[NoImage] at MeGUI.OneClickPostProcessing.createAVSFile(String indexFile, String inputFile, Nullable`1 AR, Int32 desiredOutputWidth,
Boolean signalAR, LogItem _log, AviSynthSettings avsSettings,
Boolean autoDeint, VideoCodecSettings settings, Nullable`1& dar, Boolean autoCrop, Boolean keepInputResolution, Boolean useChaptersMarks)
----[NoImage] at MeGUI.OneClickPostProcessing.StartPostProcessing()

Won't let me encode either

--[Error] [12/4/2019 6:04:14 AM] Error starting job
---[Error] [12/4/2019 6:04:14 AM] Exception message
----[NoImage] Calling setup of processor failed with error 'The file Z:\projectbackup\megui\work\02lfrdg1.uif\00002.m2ts.avs cannot be opened.
----[NoImage] Error message for your reference: [Fatal]: Failed to avformat_open_input.
----[NoImage] (Z:\projectbackup\megui\work\02lfrdg1.uif\00002.m2ts.avs, line 2)'
---[Error] [12/4/2019 6:04:14 AM] Stacktrace: at MeGUI.core.gui.JobWorker.startEncoding(TaggedJob job)
---[Error] [12/4/2019 6:04:14 AM] Inner exception: null

Line 2 is simply this: LWLibavVideoSource("Z:\projectbackup\megui\work\02lfrdg1.uif\00002.m2ts.lwi")

I have no issues when using r929 version (http://avisynth.nl/index.php/LSMASHSource) except for the above mentioned. Maybe I'm doing something wrong but reading the log from what little I understand perhaps its at my end and I don't know how to solve it in MeGUI since it asked me awhile back to end the avs script with YV12 and its coming back to haunt me or I'm also using HBD with fake lsb with SetFilterMTMode + Prefetch(1) with 720p 4:4:4 HEVC 10 bit or a combination of it all.

As mentioned the pipeline for the fake lsb uses that too to both get around having to use --y4m (https://x265.readthedocs.io/en/latest/cli.html#cmdoption-y4m) and I dunno how to go about it for my script without it as HEVC won't let me as I don't know any other method or I just am not aware of it.

If anybody can let me know if I am doing something wrong. If I can't update I'll be fine using r929 version (http://avisynth.nl/index.php/LSMASHSource). I don't wanna change my script if possible. :thanks:

[StainlessS]

Nice catch HW :)

[Pat357]

It was my own stupid fault :
I used LibavSMASHSource instead of LWLibavSource : this can never work for a webm container !!

My apologies for my mistake.

[dREV]

Don't append .lwi in your source.

:scared: Can you help me understand what you mean? Because this 00002.m2ts.lwi is created by the MeGUI program itself.

The MeGUI program has a avisynth configuration dialog field that has already <input> then under it is where I place my script and the program does the rest all via one-click.

I don't know if I can do anything within the <input> code.

From the Wiki (http://avisynth.nl/index.php/LSMASHSource#Archived_Downloads) I've tried all previous version I was able to download and ver r935+26-20190712 (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases) or whatever version this is was able to work. The rest do the same error as the current version but with some difference.

[LigH]

Always use the name of the media file in LwLibavVideoSource. In MeGUI, open the media file in the dialog, not the index file.

If MeGUI prefers to load the index file instead of the media file in a specific operation mode, it is an issue in MeGUI and should be discussed in its related threads, with all possibly relevant details, how you use it, step by step, click by click...

You can still manually edit the generated AviSynth script in MeGUI before you add a task to its queue, to possibly walk around this issue, if you don't use it in a fully automated mode.

[MeteorRain]

A quick guess is the plugin used to read the media file inside lwi. Now the field is gone (why not), it's not possible to find the media file by index file.

So please always use media file as the source. lwi is just an index.

[dREV]

Always use the name of the media file in LwLibavVideoSource. In MeGUI, open the media file in the dialog, not the index file.

If MeGUI prefers to load the index file instead of the media file in a specific operation mode, it is an issue in MeGUI and should be discussed in its related threads, with all possibly relevant details, how you use it, step by step, click by click...

You can still manually edit the generated AviSynth script in MeGUI before you add a task to its queue, to possibly walk around this issue, if you don't use it in a fully automated mode.

The programs seems to want to load the index file instead then starts encoding from the source. I've tried to edit the avs script to target the source but not sure if I did it wrong or if MeGUI won't budge. The latest version of MeGUI still uses the same version from 2017 which takes a while to index compared to both HolyWu and MeteorRain's versions. The one that worked as noted above takes almost a minute which is way better but would like to use HolyWu for that 1 frame fix.

I'll ask the author of MeGUI and see if any replies.

A quick guess is the plugin used to read the media file inside lwi. Now the field is gone (why not), it's not possible to find the media file by index file.

So please always use media file as the source. lwi is just an index.

I can't do anything on that. I do everything via One-Click.

There is another method to encode using that program but I have zero experience how to go about it which is the first image on the official site https://sourceforge.net/projects/megui/ I have no idea how to encode without MeGUI doing its thing with the indexing in One-Click. If I can be given an example for me to try it out.

Is it just: Load l-smash > LWLibavVideoSource("path\00002.m2ts") ? Then the rest of my script?
Thanks for the replies.

[hello_hello]

Having read the last few posts....

MeGUI has been using the index file as the source for LWLibavVideoSource for as long as I can remember, just as it does for d2v index files. The Lsmash index file previously contained the path to the source file, so using it as the source in MeGUI's script creator worked as it does for DGIndex.
That's no doubt the issue, because MeGUI hasn't caught up to the change yet, which in my opinion is a step backwards. In the case of MeGUI, the working directory can be different to the directory where the source file is located, so opening the index file directly has always allowed the source to be re-opened in a new script by opening the index file, without MeGUI wanting to index it again, no matter where the index file is located (existing DGIndex index files can be used as the source for the same reasons). There's no way to manually type the source name/location when creating a new script via the script creator. You have to open a source file or an existing index file, and MeGUI creates a script using the index file as the source after indexing with Lsmash, or it expects to be able to open an existing one. It's a pity newer Lsmash versions don't work the same way, IMHO.

For ffms2, the path to the source isn't saved to the index file, but MeGUI still allows you to open the index file directly, however I think it must have the same name as the source file including the original extension, followed by the ffindex extension (that's how MeGUI creates the index file anyway). MeGUI then knows to create a script using the original file name for the source with the ffms2 cachefile argument pointing to the existing index file, rather than having to repeat the indexing process. It's not as convenient as indexing with DGIndex or Lsmash (once) was though, if you want to create a new script from scratch, because the index file has to be located in the same directory as the source file for it to work.
Does the newer Lsmash have the same cachefile argument, or is MeGUI going to be forced to re-index a source file to open it it in a new script (or maybe taught to check for an existing index file)?

One disadvantage of indexing with Lsmash in the past is it hasn't been possible for MeGUI to tell Lsmash not to index the audio when it's not necessary, which it can for ffms2. I also recall reading that this can now be done for newer LSmash versions, so it's no doubt another change that will require MeGUI to be updated for in the future.

PS I assume MeGUI always includes the original extension in the script and index file names to avoid conflicts when the output directory and the source directory are the same. ie for a source file named video.mkv the script would be named video.mkv.avs, the index file video.mkv.lwi, and the output file video.mkv.mkv.

[MeteorRain]

You have to open a source file or an existing index file, and MeGUI creates a script using the index file as the source after indexing with Lsmash, or it expects to be able to open an existing one.

I don't use MeGUI and I never thought this was even a problem. During my life I never tried putting lwi in the source position, hence I had the wrong assumption.

So how does it index the source then. Usually I put the media in the source position, you know, lwlibavvideosource(media), load it in some sort of GUI and the indexing is done. You say that MeGUI creates index first and then creates the script? I'm very confused now.

[hello_hello]

I don't use MeGUI and I never thought this was even a problem. During my life I never tried putting lwi in the source position, hence I had the wrong assumption.

So how does it index the source then. Usually I put the media in the source position, you know, lwlibavvideosource(media), load it in some sort of GUI and the indexing is done. You say that MeGUI creates index first and then creates the script? I'm very confused now.

I don't use MeGUI's OneClick encoder as a previous poster does, but doing it the "manual" way you can either open a file with MeGUI's File Indexer and create an indexing job (the File Indexer lets you choose the indexer and also extracts the audio or creates a script to re-encode it as required), or you can open the source file directly with MeGUI's Script Creator, in which case MeGUI will open the File Indexer to index it first anyway. In the case of LSmash, once the indexing job has run it moves the index file to the working directory (if one is specified in it's options) and then opens the Script Creator using the index file as the source. There's also the option to run a batch of indexing jobs without opening the Script Creator each time, in which case the index files can be used as the source for the Script Creator later on.
I don't know exactly how MeGUI's File Indexer tells Lsmash to index a source, as it does that "behind the scenes", but the log file indicates it creates a script to open the source file which it runs, causing Lsmash to index.

The initial script created by MeGUI's Script Creator after indexing would look like this:
LWLibavVideoSource("D:\Episode 4.mkv.lwi")

It's always worked much the same way as indexing with DGIndex, where the index file becomes the source.
DGDecode_mpeg2source("D:\Episode 4.d2v")
For ffms2, I assume MeGUI uses the cachefile argument to create the index file in the working directory.

I checked and if you open a source file directly using the script creator and an lwi index file exists in the same directory, MeGUI appears to verify it's valid and uses the source file as the source in the script, rather than the index file. At least for MKVs. I haven't checked other file types, but for file types where it'd normally index with a different indexer by default, it'd probably insist on indexing again, so being able to open an index file directly would prevent that. There's no way to bypass the File Indexer completely if MeGUI thinks a source needs indexing though.

A way around the problem when using a newer version of Lsmash would be to manually create a script to open the source file, and to use that script as the source for MeGUI. When it's opened, Lsmash would automatically index the source file and MeGUI will patiently wait for the indexing to finish. Scripts can also be used as source files for the One Click encoder and batch encoding, but because MeGUI isn't seeing the source directly it can't know if it's anamorphic, and bypassing the File Indexer that way would mean the user would have to extract the audio themselves if they don't wish to re-encode it.

Not including the path to the source in the index file probably isn't the end of the world, but MeGUI will have to be changed for Lsmash so it doesn't use the index file as the source, and I assume the index file is always going to have to live in the source directory.

[dREV]

I don't use MeGUI and I never thought this was even a problem. During my life I never tried putting lwi in the source position, hence I had the wrong assumption.

Why does your compiled version r935+2 (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases) still worked in MeGUI but updated iterations from r935+26-20190811 (http://avisynth.nl/index.php/LSMASHSource#Archived_Downloads) compiled by HolyWu to the latest don't?

Was this .lwi taken out of the code or something? Is it possible to put whatever got it working to the latest lsmash? Perhaps then would the MeGUI author be willing to finally update. :p

[hello_hello]

Why does your compiled version r935+2 (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases) still worked in MeGUI but updated iterations from r935+26-20190811 (http://avisynth.nl/index.php/LSMASHSource#Archived_Downloads) compiled by HolyWu to the latest don't?

Was this .lwi taken out of the code or something? Is it possible to put whatever got it working to the latest lsmash? Perhaps then would the MeGUI author be willing to finally update. :p

The release info says it happened with version r935+31-20190820
http://avisynth.nl/index.php/LSMASHSource#Archived_Downloads
Remove InputFilePath field from the index file. It's unnecessary and troublesome when users rename or move the source file.

Given it doesn't seem to be common practice to use the index file as the source in scripts, I'm not sure how including the source path could be much of a problem. If you moved or renamed the source file and opened it in a script, wouldn't lsmash just index it again anyway? And if you are using the index file as the source... don't move or rename the source file. :)

The ffms2-like cachefile argument is a welcome addition, although I've still found it a bit of an annoyance because you can't re-open the index file directly (with MeGUI) unless it's in the same directory as the source.

DGIndex has worked that way forever. The path to the source is stored in the index file and it becomes the source for DGDecode. Maybe a new argument to make including the source path optional might be a compromise? Obviously LWLibavVideoSource has the ability to open an index file, so it seems a pity to lose that completely when it'd often be useful.

[hello_hello]

Use a more maintained GUI other than MeGUI. End of the story.

So that's why not writing the source path to the index file is an improvement rather than a step backwards... because MeGUI isn't always updated quickly. I'd not seen the obvious connection until now.

[Atak_Snajpera]

So that's why not writing the source path to the index file is an improvement rather than a step backwards... because MeGUI isn't always updated quickly. I'd not seen the obvious connection until now.

Index file should not contain hardcoded Path to a file. End of the story.

[manolito]

Index file should not contain hardcoded Path to a file. End of the story.

Says who? Are there any coding guidelines which tell this?

LSmash is just following the good old DGIndex example, and I never heard any complaints about the d2v file format. I just rechecked with the latest DGIndex version:
DGIndexProjectFile16
1
F:\Download\DVD_test.mpg

[filler56789]

Says who? Are there any coding guidelines which tell this?

LSmash is just following the good old DGIndex example, and I never heard any complaints about the d2v file format. I just rechecked with the latest DGIndex version:

DGIndexProjectFile16
1
F:\Download\DVD_test.mpg


Atak has a point; an index file that does not hardcode the path to the source-file implies ~portability~. By the way, a .GRF file generated by GraphEdit or GraphStudio is not an index of course but it also breaks the portability of the source-file(s) because of the hardcoded paths.

[videoh]

You can omit full paths too with DGIndex, leaving just the bare filename, which makes things relative to the current directory, and such that the files can be moved together to a new location without needing re-indexing.

[hello_hello]

Says who? Are there any coding guidelines which tell this?

LSmash is just following the good old DGIndex example, and I never heard any complaints about the d2v file format. I just rechecked with the latest DGIndex version:

That's why I've mentioned DGIndex/DGDecode in a few posts, thinking from there it'd be natural to question why it's a problem for LSMash but not DGIndex. Instead there's been two posts claiming "end of story" status without explaining why it's a bad thing, or why an option to enable/disable it wouldn't keep everyone happy (and it'd allow Lsmash to keep playing nice with GUIs).

The cachefile argument is nice to have, but from the perspective of someone who uses a GUI, the ability to specify it's location isn't as useful when the index file doesn't know where the source is, as it can't be opened directly later on unless it's in the source folder. Could we have a show of hands from those who often use the cachefile argument without specifying a location other than the source folder? Does anyone regularly give index files fun names while still writing them to the same folders as the source?

What about something like this? If the cachefile argument isn't used, or of the index file is written to the source folder, by default LSmash would not save the full path to the index file. If the index file is not written to the same folder as the source, then by default LSmash would write the full path. A FilePath argument would allow the user to over-ride those defaults. Just a thought...

[StainlessS]

Embedding the filename in the index may or may not have been a good idea to begin with, but breaking things without good reason is also not a good idea.

EDIT: Is/was also a lousy (and lazy) idea for MeGUI to use embedded filename to load source, I was a bit shocked to discover that it did that.

[hello_hello]

EDIT: Is/was also a lousy (and lazy) idea for MeGUI to use embedded filename to load source, I was a bit shocked to discover that it did that.

I don't know if I'd call it lazy, because the path is saved to the index file, and LWLibavVideoSource can obviously open index files, and I suspect that's by design rather than accident. but if the index file wasn't located in the source folder there was no way to tell LWLibavVideoSource where to find it, until recently. Lsmash has only had a cachefile argument for roughly 15 minutes. Why is it considered normal for DGDecode to load d2v files but evil for everyone else?

While I'm defending MeGUI....
If I remember correctly adding Lsmash as an indexer wasn't completely straightforward. I don't think older versions report the indexing progress because I recall Zathor saying he had to find a way to let MeGUI know when it finished, and MeGUI had to be taught to move the index files to the working directory so lsmash wouldn't be embarrassed in the company of cachefile enabled indexers.

So.... the index files are in the working directory and it's time to create a new script from scratch. MeGUI can open d2v index files and until recently it worked the same way for lsmash, but not so much for ffms2 because the index file has no idea where the source file is. You can work around it by manually moving/copying the index file to the source directory and open it as the source in MeGUI's script creator. MeGUI will create a new script with the index file specified for the cachefile argument. I assume the index file must be named correctly for MeGUI to get the source name right.

[Atak_Snajpera]

Says who? Are there any coding guidelines which tell this?

Mr. Common sense my friend... No guidelines needed.

[hello_hello]

Mr. Common sense my friend... No guidelines needed.

Does the source path saved to the index file play any part in determining if the source and index file are a match? I assume not, given it's no longer written to the index file, so would it be logical to assume it only tells Lsmash where to find the source that's expected to match?

My questions are.... well they're not actually my questions, but Mr. Common Sense will hassle me relentlessly until I ask on his behalf.....

Mr. Common Sense is keen to understand why the source/path info in the index file couldn't be ignored when LWLibavVideoSource is opening a video file directly, and why it wouldn't function exactly as it does without it. He's quite adamant the freedom to rename or move files won't change whether the source/path info in the index file is ignored or it's not written-in the first place.

Mr. Common Sense is insisting if the source info written to the index file is only used when an index file becomes the source for LWLibavVideoSource it's so obviously common sense that the index file should include the video file's location, not including it becomes borderline mental. There's no cachefile-equivalent argument for specifying the video file's location when opening index files.

Mr. Common Sense wants me to emphasise his/my entire post is purely a common sense view of how the world should work, with possibly no basis in reality, due to common sense being so uncommon. :)

[Atak_Snajpera]

Does the source path saved to the index file play any part in determining if the source and index file are a match? I assume not, given it's no longer written to the index file, so would it be logical to assume it only tells Lsmash where to find the source that's expected to match?

My questions are.... well they're not actually my questions, but Mr. Common Sense will hassle me relentlessly until I ask on his behalf.....

Mr. Common Sense is keen to understand why the source/path info in the index file couldn't be ignored when LWLibavVideoSource is opening a video file directly, and why it wouldn't function exactly as it does without it. He's quite adamant the freedom to rename or move files won't change whether the source/path info in the index file is ignored or it's not written-in the first place.

Mr. Common Sense is insisting if the source info written to the index file is only used when an index file becomes the source for LWLibavVideoSource it's so obviously common sense that the index file should include the video file's location, not including it becomes borderline mental. There's no cachefile-equivalent argument for specifying the video file's location when opening index files.

Mr. Common Sense wants me to emphasise his/my entire post is purely a common sense view of how the world should work, with possibly no basis in reality, due to common sense being so uncommon. :)
You are barking under wrong tree. You should bark under MeGUI tree...

[hello_hello]

You are barking under wrong tree. You should bark under MeGUI tree...

It's not about MeGUI. It's about fixing something in lsmash that wasn't broken (in my opinion).
Mr. Common Sense is beginning to suspect if there's a good argument for never writing the source path to the index file, someone would've offered it by now, or maybe explained why I'm wrong for thinking it should at least be optional, or something....

On a completely different subject, the usual expression is "barking up the wrong tree (https://en.wikipedia.org/wiki/Barking_up_the_wrong_tree)", in case you weren't aware. Unless that's an alternative version I've not heard before. :)

[Are_]

Nothing is broken in lsmash, only something is broken in megui if you try to use an unsupported fork of that plugin on that gui. Is as simple as using another fork, or asking the megui author for an update.

[hello_hello]

I don't know why or what you are insisting upon.

There's no insisting taking place. I'm just waiting for someone to answer a single question I've asked or to explain what problems were fixed by the change. I'm starting to accept that's not realistic, as even beginning my last post with "it's not about MeGUI" didn't help.

FFMS2 also doesn't write the source path to the index file. Does any of the FFMS2/MeGUI users (except you) complain or get annoyed because he/she couldn't directly use the index file as the source for whatever ridiculous reasons?

I've explained how MeGUI works around that shortcoming, but as long as the index file is in the same folder as the source you can "open it" to create a new script. A GUI can't add the cachefile argument to a script if the index file isn't in the source folder because it can't know it exists, and if it is in the source folder there's probably no need to use the cachefile argument anyway.

It's simply that an user updated a plugin on his/her own but MeGUI hasn't been updated yet to keep up with the latest changes. Since MeGUI can use the media file as the source and make use of the cachefile argument for FFMS2, the developer of MeGUI can just do the same thing as well for lsmash.

Obviously.... and someone donating their time to update a program or plugin is something I appreciate even when they've changed something I'd prefer they hadn't, and even when the change forces someone else to donate their time to accommodate it. I mostly use a PC running XP and an old version of lsmash, so I only posted here because I stumbled across the lsmash/MeGUI question and the replies seemed to indicate it's not common knowledge lsmash could open index files directly. I still think removing that ability completely was the wrong choice, but for some reason everyone wants to talk about MeGUI instead.

[hello_hello]

The index file is rebuilt needlessly due to InputFilePath field when the users move/rename the files or the folder containing the files to another folder/drive, or when open the media file on another computer via network path. It simply offers no real benefits at all.

Do you mean aside from allowing you to open the index file directly? Was it not possible to open the index file in notepad to change the InputFilePath field after renaming or moving files?

Because the author of lsmash never formally documented it in the readme to tell the users to open the index file directly.

That does seem odd, but it explains how the change could've been made without knowing it'd break something.

Please tell me under what circumstances would an user bother to take another unnecessary step to modify the script and append .lwi at the end of source path just for LWLibavVideoSource to specifically open the index file while it already could directly open the media file just fine.

In that scenario, I can't think of a reason to do so.

You kept yelling how useful that ability was for everyone

Are you mistaking calmly answering questions for yelling or are you being childish?

but actually the point was only meant to satisfy the bad design of MeGUI. Fortunately we have other more actively developed/maintained GUIs in the forum and the users can have better choices.

Which other GUI's are you referring to? I'd like to try them. I assume you're referring to GUI's that index a source but don't repeat the process if you open it a second time. Which one would you recommend and how does it work in that respect?

I use AVISynthesizer quite a bit for creating scripts. In case you're not aware of it, you create templates to use via Explorer's right click menu and AVISynthesizer adds the path/source details and creates the script. When the index file wasn't in the source folder you could previously use it instead of the source, or even create a new script manually without requiring any tedious typing for the cachefile argument, and even when doing so to open it in avspmod.
Admittedly, that sort of thing only qualifies as a minor annoyance, but we all do things differently. Not everyone creates a script to open and index a source in avspmod before moving or renaming the source file.

I honestly don't care that much if lsmash remains the way it is, but the refusal to acknowledge there might have been a better way to solve the source renaming/moving problem does baffle me a little, as does attempting to blame a GUI or label it as badly designed, especially if the change was made without knowing index files could be used as the source.

[Are_]

I know I already bit the bait but, shouldn't we ignore the obvious troll? He doesn't use the plugin but he's just wasting others time with ridiculous questions nobody cares just because he's probably bored and all. In the last post he even retorted to mild insulting, sad view.

[LigH]

I'm not fond of religious wars in technical forums.

[hello_hello]

I know I already bit the bait but, shouldn't we ignore the obvious troll? He doesn't use the plugin but he's just wasting others time with ridiculous questions nobody cares just because he's probably bored and all. In the last post he even retorted to mild insulting, sad view.

FFS, that'll teach me to post about something that doesn't effect me at the moment simply to help someone else understand a problem they were having. I do use the plugin, but I don't use the current version and I made the reason for that clear. I'm sorry you didn't understand... and all... but in case I need to state the obvious, I'd prefer to eventually use a more recent version, only I'd rather it retained the ability to open index files.

For the record, it's not the 1990's and the only trolls in forums are the ones who troll by insulting others with accusations of trolling. Did you forget to discuss the topic or were you just bored?

[videoh]

Suppose you want to load a series of M2TS files which are scattered in a directory and not in sort order (just about any bluray/UHD disk). Another case is multiple VOBs from a DVD. Loading a DGIndex(NV) project file will conveniently and automatically load them all in order, because the file list is stored in the index file. Without that you'd have to navigate to and tediously select all the files again every time you want to work on the project again. Is this something applicable here? If smash whatever can only load one source file then it would be irrelevant. I don't know enough about the smash stuff to know if this is relevant, but it sure is a big advantage for DGIndex(NV) to have that capability.

[videoh]

I'm not fond of religious wars in technical forums. LigH arrogantly thinks that what he is fond of is an important consideration. Newsflash, nobody cares what you are fond of.

[MeteorRain]

Sorry for not keeping an eye on this thread for a few days. Let me add some point from my view.

The fundamental difference between dgindex / dgnv and lsmash / ffms is that dg index is generated using a standalone tool while lsmash / ffms index is (usually) generated by the plugin itself.

That's why I was so confused by the behavior of MeGUI. You have to use the source file as the parameter passed to lsmash in order to create an index. Why would it change that parameter (completely unnecessarily) to the index then?

For dgindex and dgnv it's pretty natural to use index because you create the index first. Besides, as videoh said, dg family supports loading multiple files in a single shot, which makes more sense for it to load an index.

[hello_hello]

Suppose you want to load a series of M2TS files which are scattered in a directory and not in sort order (just about any bluray/UHD disk). Another case is multiple VOBs from a DVD. Loading a DGIndex(NV) project file will conveniently and automatically load them all in order, because the file list is stored in the index file. Without that you'd have to navigate to and tediously select all the files again every time you want to work on the project again. Is this something applicable here? If smash whatever can only load one source file then it would be irrelevant. I don't know enough about the smash stuff to know if this is relevant, but it sure is a big advantage for DGIndex(NV) to have that capability.

I don't actually know if lsmash can open split vob files as a single file as I've always used DGIndex for that, but it's the same as ffms2... by default it ignores repeat flags and outputs the average frame rate. It's probably fine if the ultimate goal is a variable frame rate encode, but I've seen a few threads at VideoHelp where someone has ripped a DVD with MakeMKV and indexed with lsmash and posted to ask why the audio sync is all over the place.

As far as I know there's no way to load multiple sources as a single index file or project.

[hello_hello]

That's why I was so confused by the behavior of MeGUI. You have to use the source file as the parameter passed to lsmash in order to create an index. Why would it change that parameter (completely unnecessarily) to the index then?

I can only repeat what I've repeated several times. The main advantage of being able to open an index file directly is when it's not in the source folder. That's why MeGUI works the way it does. It can index the source for you Monday, move the index file to the working directory, and Wednesday you can open it to create a script. It couldn't work that way for ffms2, but Lsmash had the ability to open index files and the person maintaining MeGUI was clever enough to make use of it. It baffles me why anyone would think that's a bad thing.

If lsmash wrote the path to the index file while indexing, but ignored it unless an index file is used as the source, it seems to me it'd be the best of both worlds. I honestly don't understand why there's so much resistance to the idea. Why break functionality unnecessarily? Even if the response was "can't be bothered", or "don't care" it'd make more sense to me than denying there's ever a reason to open index files. I often do because I rarely keep them in the source folder, so it saves having to specify each source location plus the cachefile argument for the index file in a new script, and we already know one of the most popular GUIs opens index files and it'll break.

[kedautinh12]

So how did you right click on a nonexistent index file in the explorer when the index wasn't created yet for new media files? Didn't you still need a script and point the media file as source to let LWLibavVideoSource create the index file first? Then you either modify the existing script or create another new script and point the index file as source for LWLibavVideoSource just because you want to directly open the index file so deadly.

I see you hardwork with l-smash source, can you continute with ffmpegsource??

[hello_hello]

Of course it was. That's what the users did before, not only because of the file/folder being moved/renamed for some reasons, but also the file being opened from another computer using network path. But why must the users bother to edit the index file when they can avoid this unnecessary step once and for all?

I didn't say they should have to. Not that I recall complaints from users who move/rename source files after indexing, so I'm not sure how it was determined they found it a problem or troublesome.

So how did you right click on a nonexistent index file in the explorer when the index wasn't created yet for new media files? Didn't you still need a script and point to the media file as source to let LWLibavVideoSource create the index file first? Then you either modify the existing script or create another new script and point to the index file as source for LWLibavVideoSource just because you want to directly open the index file so deadly.

After explaining numerous times?
At the risk of bringing MeGUI into it again, I often add a batch of indexing jobs to the job queue, let MeGUI run them while moving the index files to the specified location(s), and later I create the scripts from scratch. There's no scripts to copy or modify. MeGUI does the indexing "behind the scenes" via a command prompt... or something. You can't do the same thing with ffms2 unless the index files are in the source folder, which is often a determining factor for me when choosing which indexer to use.
I often use Avisynthesizer to create scripts using the index file as the source, and sometimes I do that rather than copy or modify an existing script because it's more convenient, or sometimes I get MeGUI to run a batch of indexing jobs and create the scripts myself etc.

And to go around in circles again, MeGUI is designed so the user isn't required to have any knowledge of Avisynth including how to open a source in a script, and I can't think of a GUI of that type that doesn't require the user to open a source file. I did ask for examples of the GUIs you claimed to be better choices with no response, but in MeGUI's case, the Script Creator can add the cachefile argument as long as it's configured to automatically open after indexing. To open a source that's already been indexed, the index file must be in the source folder or of a type the Script Creator can open directly, otherwise MeGUI can't know it exists. Are there psychic GUIs I'm unaware of, or is a bad thing MeGUI allows you to specify a working directory while still avoiding re-indexing whenever possible?

I've offered several suggestions as to how I think it'd be possible to have the best of both worlds, but they've been ignored in preference to insisting everybody does things the same way, or labelling a GUI as badly designed for having the audacity to use existing functionality, and that's something I find a little hard to understand.

[dREV]

MeGUI has a field section called "Working" which is where index files goes. It doesn't have to be set to target to a specific folder as it'll create a folder with the indexed file where ever the m2ts file is without moving it which is its default. However, even with leaving it on its default using the latest lsmash it'll still give the errors I've pointed at #1085 (https://forum.doom9.org/showthread.php?p=1892043#post1892043). Image here https://i.ibb.co/4jSWfZh/vampi.jpg

I also agree that it's not cool to call MeGUI out of date when it's been using something that was included in lsmash. Somehow the author knew about it but was taken out cuz the updaters didn't know why it was there or something. So if the MeGUI author wants to resort to having to update lsmash I'll be fine using MeteorRain's working one till then despite not being able to use that 1 frame fix. :)

Suppose you want to load a series of M2TS files which are scattered in a directory and not in sort order (just about any bluray/UHD disk). Another case is multiple VOBs from a DVD. Loading a DGIndex(NV) project file will conveniently and automatically load them all in order, because the file list is stored in the index file. Without that you'd have to navigate to and tediously select all the files again every time you want to work on the project again. Is this something applicable here? If smash whatever can only load one source file then it would be irrelevant. I don't know enough about the smash stuff to know if this is relevant, but it sure is a big advantage for DGIndex(NV) to have that capability.

I don't actually know if lsmash can open split vob files as a single file as I've always used DGIndex for that, but it's the same as ffms2... by default it ignores repeat flags and outputs the average frame rate. It's probably fine if the ultimate goal is a variable frame rate encode, but I've seen a few threads at VideoHelp where someone has ripped a DVD with MakeMKV and indexed with lsmash and posted to ask why the audio sync is all over the place.

As far as I know there's no way to load multiple sources as a single index file or project.

Would that even work when using trim? When I find banding sections I have to do multi-uses of the same m2ts file (banding here, sharpening here, anti-aliasing here, etc) and that would be really helpful cuz it takes a loooong time (well the 2017 lsmash MeGUI was using but even with MeteorRain's working one still takes time) but I guess it won't if I use trim? I don't really know what the index does. Does it index the entire m2ts file or is it frame based indexing?

I have my MeGUI setup to use lsmash via One Click > Config > Other tab > Indexer / Opener Priority and seems to ignore those and goes directly to DGI. To nitpick on this I dunno why it varies between DVD's with regards to how it does VOBs because I've had some DVD's where it only did each single episode while in others it did the entire DVD episodes into one huge container. :eek:

[hello_hello]

MeGUI has a field section called "Working" which is where index files goes. It doesn't have to be set to target to a specific folder as it'll create a folder with the indexed file where ever the m2ts file is without moving it which is its default. However, even with leaving it on its default using the latest lsmash it'll still give the errors I've pointed at #1085 (https://forum.doom9.org/showthread.php?p=1892043#post1892043). Image here https://i.ibb.co/4jSWfZh/vampi.jpg

I don't use OneClick, but I assume MeGUI still uses the index file as the source for lsmash in the script even when it's in the source folder because there's previously been no reason not to.

Would that even work when using trim? When I find banding sections I have to do multi-uses of the same m2ts file (banding here, sharpening here, anti-aliasing here, etc) and that would be really helpful cuz it takes a loooong time (well the 2017 lsmash MeGUI was using but even with MeteorRain's working one still takes time) but I guess it won't if I use trim? I don't really know what the index does. Does it index the entire m2ts file or is it frame based indexing?

It'd be better to discuss this elsewhere if need be, but I'm not sure exactly what you mean anyway. I'll send you a PM shorty so we can do it that way if you like.

I have my MeGUI setup to use lsmash via One Click > Config > Other tab > Indexer / Opener Priority and seems to ignore those and goes directly to DGI. To nitpick on this I dunno why it varies between DVD's with regards to how it does VOBs because I've had some DVD's where it only did each single episode while in others it did the entire DVD episodes into one huge container. :eek:

I'm guessing, but it probably ignores the preferred indexer and uses DGIndex for vob files due to the problem I mentioned earlier.
Vob files are generally indexed as a single file when they're sequentially numbered. There's often a title for each episode, but sometimes there's just a single title with the episodes divided by chapters and that'd be why sometimes all the episodes are encoded together. It depends how the DVD is authored. I'll explain how I re-author DVDs before encoding to solve that problem in the PM.

Edit: I tried to send you a PM but you have them disabled.

[dREV]

It'd be better to discuss this elsewhere if need be, but I'm not sure exactly what you mean anyway. I'll send you a PM shorty so we can do it that way if you like.

Sorry for not writing my words more properly I hope this will do better. The index is not based on what's in the AviSynth script such as if trim were to be used, correct?

For example, if I have a banding scene from frames (200, 500) and I get lsmash to index and encode these scenes it's not really doing it for those scenes but for the m2ts file itself? I'll then have to run the lsmash index 2 more times to do frames scenes (0, 199) and frames (501, 33000) to finish up the episode.

It'll be very helpful if I can just index once and get straight to encoding especially when I use trim as it could take up to 2 to 50 different frames for just 1 single episode which could take a very long time to do. Especially on the original MeGUI lsmash which I've timed it to be over well an 1 hour or two and movies are worse as the numbers shoot even higher.

I'm guessing, but it probably ignores the preferred indexer and uses DGIndex for vob files due to the problem I mentioned earlier. Vob files are generally indexed as a single file when they're sequentially numbered. There's often a title for each episode, but sometimes there's just a single title with the episodes divided by chapters and that'd be why sometimes all the episodes are encoded together. It depends how the DVD is authored. I'll explain how I re-author DVDs before encoding to solve that problem in the PM.

Edit: I tried to send you a PM but you have them disabled.

Sorry I never expected to receive any messages. Sorry if I wasted your time if you wrote stuff via PM. I'm not looking to re-author DVD's as I just do backups in mkv. Thanks for your offer

[poisondeathray]

Sorry for not writing my words more properly I hope this will do better. The index is not based on what's in the AviSynth script such as if trim were to be used, correct?

Yes, index is for the main video


For example, if I have a banding scene from frames (200, 500) and I get lsmash to index and encode these scenes it's not really doing it for those scenes but for the m2ts file itself? I'll then have to run the lsmash index 2 more times to do frames scenes (0, 199) and frames (501, 33000) to finish up the episode.


No, you only need to index once, the whole video

[hello_hello]

For example, if I have a banding scene from frames (200, 500) and I get lsmash to index and encode these scenes it's not really doing it for those scenes but for the m2ts file itself? I'll then have to run the lsmash index 2 more times to do frames scenes (0, 199) and frames (501, 33000) to finish up the episode.

While sometimes you can't know there's problems with an encode till after it's done, that's why I don't use the OneClick encoder, as normally I'd preview the Avisynth output before encoding and try to do everything in a single script, but even if you encode the video in sections you should be able to re-use the index file. If your workflow is causing MeGUI to create a new script and re-index the source for each section you encode you should change it if you can. OneClick is really designed to be an automatic process. If you're creating complex scripts and/or fixing problems it's probably not the best choice.

LWLibavVideoSource("D:\Video.m2ts.lwi")
MaybeSomeFilterHere()
MaybeAnotherOne()
A = last
B = A.SomeDebanding()
A.Trim(0, 199)\
++B.Trim(200, 500)\
++A.Trim(501, 0).MaybeAnotherFilter().OrTwo()
MaybeSomethingElse()

By the way, I had a better look at your previous screenshot and it seems even when you don't specify a working directory, OnceClick creates a subfolder with a random name in the source directory, so it'd put the index file in that folder and open it directly. OneClick probably always creates subfolders regardless of the output location, but as I said, I don't use it.

Especially on the original MeGUI lsmash which I've timed it to be over well an 1 hour or two and movies are worse as the numbers shoot even higher.

That seems an extremely long time even for a 1080p source with several audio streams. I just indexed a 1080p movie with the old XP compatible lsmash and it took 70 seconds. It was only 6GB but it was the largest MKV I had handy. Sometimes indexing a 1080p Bluray movie can take a while, but I don't think I've indexed a movie that came close to an hour to complete.

Sorry I never expected to receive any messages. Sorry if I wasted your time if you wrote stuff via PM. I'm not looking to re-author DVD's as I just do backups in mkv. Thanks for your offer

The idea of re-authoring them first is so the episodes can be encoded individually. It's easy to do and doesn't take long.
To keep discussing MeGUI itself though, it'd probably be better to do so elsewhere rather than sidetrack this thread too much.

[Atak_Snajpera]

@HolyWu
Could you help me to decrypt those values?
Index=0,POS=9566774,PTS=-9223372036854775808,DTS=35085050,EDI=0
Key=1,Pic=1,POC=2,Repeat=1,Field=0

My guesses
Index -> stream index
POS -> position in file
PTS -> ?
DTS -> ?
EDI -> ?
Key -> keyframe
Pic -> ?
Repeat -> ?
Field -> Interlaced field

Also What is purpose of these entries?

<StreamDuration=0,0>-9223372036854775808</StreamDuration>
<StreamIndexEntries=0,0,16061>
POS=0,TS=0,Flags=1,Size=0,Distance=0
POS=12116,TS=500500,Flags=1,Size=0,Distance=0
POS=26960,TS=1101100,Flags=1,Size=0,Distance=0
POS=95503,TS=1601600,Flags=1,Size=0,Distance=0
POS=154509,TS=2002000,Flags=1,Size=0,Distance=0
POS=215660,TS=2452450,Flags=1,Size=0,Distance=0
POS=314434,TS=2752750,Flags=1,Size=0,Distance=0
POS=476442,TS=3253250,Flags=1,Size=0,Distance=0
POS=658539,TS=3753750,Flags=1,Size=0,Distance=0
POS=763885,TS=4104100,Flags=1,Size=0,Distance=0
POS=925780,TS=4654650,Flags=1,Size=0,Distance=0

[Atak_Snajpera]

They seem to be used for format which doesn't support seeking natively.
For example raw stream unmuxed in container (.264 , .265 , ivf) ?

[LigH]

Probably also MPEG Transport Stream (and Program Stream?): No internal index chunk, originally designed to be "played-a-live" ;) in a broadcast; seeking is only possible in a kind of interval bisection method: test seek with good guess assuming a rather constant average bitrate, read next found GOP timestamps, calculate new difference, new test seek with a better guess...

[Atak_Snajpera]

Probably also MPEG Transport Stream (and Program Stream?): No internal index chunk, originally designed to be "played-a-live" ;) in a broadcast; seeking is only possible in a kind of interval bisection method: test seek with good guess assuming a rather constant average bitrate, read next found GOP timestamps, calculate new difference, new test seek with a better guess...

So another question would be Why do we need those entries for mkv,mp4 and others?

[LigH]

LwLibav*Source generally does not rely on internal index chunks of any container format. It always re-indexes content streams as they are demultiplexed by the libavformats splitters. Only a fresh index guarantees frame-exact seeking; internal index chunks chould have been damaged, missing, or been built by "incompetent" software...

LSMASH*Source relies on index chunks of ISO Base Media container formats only.

[Atak_Snajpera]

LwLibav*Source generally does not rely on internal index chunks of any format. It always re-indexes content streams as they are demultiplexed by the libavformats splitters. Only a fresh index guarantees frame-exact seeking; internal index chunks chould have been damaged, missing, or been built by "incompetent" software...

I meant why do we need those extra entries in index if we already have this above

<LSMASHWorksIndexVersion=0.0.2.0>
<LibavReaderIndexFile=16>
<FileSize=2465531781>
<FileHash=0x94ac5e38>
<LibavReaderIndex=0x00000100,1,mpegvideo>
<ActiveVideoStreamIndex>+0000000000</ActiveVideoStreamIndex>
<ActiveAudioStreamIndex>-0000000002</ActiveAudioStreamIndex>
<StreamInfo=0,0>
Codec=2,TimeBase=1/1200000,Width=720,Height=480,Format=yuv420p,ColorSpace=6
</StreamInfo>
Index=0,POS=0,PTS=-9223372036854775808,DTS=0,EDI=0
Key=1,Pic=1,POC=0,Repeat=1,Field=0
Index=0,POS=7988,PTS=-9223372036854775808,DTS=50050,EDI=0
Key=0,Pic=2,POC=1,Repeat=1,Field=0
Index=0,POS=8576,PTS=-9223372036854775808,DTS=100100,EDI=0
Key=0,Pic=2,POC=3,Repeat=1,Field=0
Index=0,POS=9128,PTS=150150,DTS=150150,EDI=0
Key=0,Pic=3,POC=2,Repeat=1,Field=0
Index=0,POS=9446,PTS=-9223372036854775808,DTS=200200,EDI=0
Key=0,Pic=2,POC=5,Repeat=1,Field=0
Index=0,POS=10002,PTS=250250,DTS=250250,EDI=0
Key=0,Pic=3,POC=4,Repeat=1,Field=0
Index=0,POS=10320,PTS=-9223372036854775808,DTS=300300,EDI=0
Key=0,Pic=2,POC=6,Repeat=1,Field=0
Index=0,POS=10884,PTS=-9223372036854775808,DTS=350350,EDI=0

[l33tmeatwad]

Was just trying out the latest updates from HolyWu, apparently every version after 20190910 either crashes or gives an access violation on x86 with AviSynth+ 3.4.0; the x64 for the latest appears to work fine.

[l33tmeatwad]

I just tested it by 32-bit AVSMeter on a few clips with codecs avc hevc vc-1 mpeg2 vp9 av1, and I couldn't reproduce it.
Tried it on two different Windows 10 machines (both 64-bit Windows) and on macOS though WINE. The x64 version works on the same machines.

[l33tmeatwad]

I tried it on x64 Win10 as well. But I couldn't reproduce it due to lack of useful information. Post your full script and the mediainfo of your media file. What program did you use to open the script? Have you tried to open other media files and it's still the same?
I'll post some more information tomorrow when I'm at my computer again, but it was several MP4s containing x264 encoded video. The script was literally just LWLibavVideoSource and nothing else and works with the x64 plugin. It is not even generating the index file with the x86 version, just displays access violation in the AvsPmod preview window with no specific error codes.

[manolito]

Just like HolyWu I could not reproduce it on my machine... :confused:

Hardware:
Core i5 (two physical cores plus two hyperthreading cores)
8 GB RAM

Software:
Win7 64
AVS+ 3.40 32bit
Latest HolyWu LSMASH build from Nov 2019
Old StaxRip 32bit version as the host software
Current X264 32bit MABS build by Ligh

Source:
Concert Show by Toto
AVC video 1280x720 @25fps progressive in MP4 container

Target format:
AVC video @704x396 @25fps progressive in MKV container

AVS Script:
LWLibavVideoSource("D:\Toto.mp4", cachefile="D:\Toto temp files\Toto.lwi", format="YUV420P8")
ChangeFPS(25.00)
Crop(0,0, -Width % 4,-Height % 4)
ColorMatrix(source=0,dest=2)
RequestLinear(rlim=50, clim=50)
ConvertToYV12()
Spline36Resize(704,396)
avstp_set_threads(1)
Prefetch(4)


Two things to note about the script:
1. The ChangeFPS call right after the source filter has the only purpose to force linear access. I could have used RequestLinear instead. In MT mode this is necessary, otherwise the speed will drop to a snail's pace. (FFMS2 has the same behavior)
See here:
https://forum.doom9.org/showthread.php?p=1883754#post1883754

2. The RequestLinear call after ColorMatrix ist required to prevent crashes in MT mode. It is also necessary to force MT_SERIALIZED for ColorMatrix and RequestLinear if MT is used.
See here:
https://forum.doom9.org/showthread.php?p=1865279#post1865279
The relevant information is in the third paragraph of the post.


But with these settings the latest LWLibavVideoSource versions are absolutely stable on my machine...

[Taurus]

But with these settings the latest LWLibavVideoSource versions are absolutely stable on my machine...

Same here...
Using almost identical settings as manolito mentioned.
No crash, no nothing, pur fun :D

[l33tmeatwad]

After testing a few things I found out what I think is triggering it. Apparently files that contain cover art (iTunes) appear to crash the x86 version, a recontainered version will work fine. It's odd that it works on the x64 version but not the x86 version. The following machines were used to test with the same settings:

Machine 1
Processor: Core i7-8700 @ 3.20GHz
RAM: 16GB
Operating System: Windows 10 LTSB (2019) (64-bit)
Software: AviSynth+ 3.4.0
AvsPmod v2.6.1.1
Microsoft Visual C++ 2015-2019 Redist (14.24.28127)

Machine 2
Processor: Core i3-4160 @ 3.60GHz
RAM: 8GB
Operating System: Windows 10 LTSB (2019) (64-bit)
Software: AviSynth+ 3.4.0
AvsPmod v2.6.1.1
Microsoft Visual C++ 2015-2019 Redist (14.24.28127)

Machine 3
Processor: Core i5 @ 2.5GHz
RAM: 8GB
Operating System: macOS 10.11.6
Software: PlayOnMac (WINE, System Version 3.9)
AviSynth+ 3.4.0
AvsPmod v2.6.1.1

Media File
General
Complete name : 8bit Sample.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 60.5 MiB
Duration : 3 min 13 s
Overall bit rate : 2 616 kb/s
Cover : Yes

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3 min 13 s
Bit rate : 2 440 kb/s
Width : 960 pixels
Height : 720 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.147
Stream size : 56.4 MiB (93%)
Writing library : x264 core 133 r2334 a3ac64b
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 /
psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 /
trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 /
chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 /
nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 /
bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 /
open_gop=0 / weightp=2 / keyint=240 / keyint_min=23 / scenecut=40 /
intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.60 /
qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=17500 / vbv_bufsize=17500 /
crf_max=0.0 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 3 min 13 s
Duration_LastFrame : -19 ms
Bit rate mode : Constant
Bit rate : 169 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 3.90 MiB (6%)
Default : Yes
Alternate group : 1

AviSynth Script
LWLibavVideoSource("8bit Sample.mp4", format="YUV420P8")

[Taurus]

@l33tmeatwad:
I can confirm your findings.
Any other: directshowsource, dss2, ffvideoSource, etc. is doing fine.
This is on Win7_64bit with avisynth_32bit.
Have'nt tested avisynth_64bit.
Thanks for sharing your insight...

[l33tmeatwad]

Awesome, thanks for the speedy fix!

[Taurus]

:thanks:Thank you!

[outhud]

I want to compile L-SMASH-Works for Vapoursynth on Linux. Which repository should I use for this? Are there compile instructions somewhere?

[Are_]

https://github.com/HolyWu/L-SMASH-Works/

git clone https://github.com/HolyWu/L-SMASH-Works
cd L-SMASH-Works/VapourSynth/
mkdir build
meson . build/
ninja -C build/


https://github.com/l-smash/l-smash is a dependency for the vapoursynth plugin, besides ffmpeg and vapoursynth itself.

[outhud]

https://github.com/HolyWu/L-SMASH-Works/

git clone https://github.com/HolyWu/L-SMASH-Works
cd L-SMASH-Works/VapourSynth/
mkdir build
meson . build/
ninja -C build/


https://github.com/l-smash/l-smash is a dependency for the vapoursynth plugin, besides ffmpeg and vapoursynth itself.

Thanks. I've installed ffmpeg, vapoursynth and l-smash. It seems ninja fails on the last step for L-SMASH-Works with:

/usr/bin/ld: /usr/local/lib/liblsmash.a(dts.o): relocation R_X86_64_PC32 against symbol
`lsmash_remove_dts_reserved_box' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

[Are_]

Try to recompile l-smash with
./configure --disable-static --enable-shared

[outhud]

Try to recompile l-smash with
./configure --disable-static --enable-shared

That did it! Thanks.

[poisondeathray]

Lsmash seek /refresh issue with CPU decoding

If you seek to frame 240, or 480, or 720 then push refresh, the frame jumps to frame 0

This was posted from another forum;
https://www.mediafire.com/file/pvojgkqxu4t8mii/20151224_145719.m2ts/file


-Affects avspmod refresh, or vdub2 refresh, or vsedit refresh
-Affects x86/x64 , threads=1, avs/vpy versions, several different lsmash versions

-Decodes ok with ffms2

-Decodes ok with Lsmash decoder="h264_cuvid" (!) . But "wrong" framerate @ 59.94 . Correct frames without refresh bug. If you AssumeFPS(30000,1001), or repeat=true with selecteven() or selectodd(), it works work

-Only occurs on certain P-frames ; eg. if you go to frame 44 and refresh, it does not occur


Could it be related to frame doubling flag ? This from a Panasonic ZS7 / TZ10 that shoots 29.97p with frame doubling

Frame mode : Frame doubling

Problem still persists with repeat=true and selecteven() or selectodd()


Something to do with timecodes ? There is a discrepancy between VFR/CFR time reported by ffms2 ffinfo

[manolito]

[URL]
LWLibav: Change the default of 'repeat' to true.


Thanks a lot for the update...

Still I think that changing the default for the "repeat" parameter from false to true is not such a good idea. I do not dispute that setting it to true makes sense, my concern is that this change breaks almost all of my LSMASH based scripts.

So far the source filter behavior for all my filters has been like this:

1. MPEG2Source (and all DG source filters) honor the pulldown flags by default.

2. FFMS2 and LSMASH do not honor the pulldown flags by default.

3. DirectShow based source filters also do not honor the pulldown flags, but without specifying the fps parameter they output 29.97 fps, but not by honoring the pulldown flags, but by inserting duplicate frames instead.


I would really appreciate if all the source filter authors could agree on using the same method for treating pulldown flags - once and for all...


Cheers
manolito

[manolito]

Yes, I know that...

But:
This only applies if I do not do a VFR to CFR conversion by specifying fpsnum and fpsden (at least this is what the readme says). And since all my target formats require CFR I ALWAYS try to specify fpsnum and fpsden, even if the source already is CFR. If this is not possible because the host application does not expose the values for fpsnum and fpsden, I use ChangeFPS(23.976) right after the source filter call to do the VFR to CFR conversion.

And this is where the new default behavior breaks my scripts. Up to now with a soft telecined source LSMASH would spit out 23.976 fps, and the ChangeFPS command would make sure that I get perfect A/V sync (maybe at the cost of some duplicate or dropped frames). The new default behavior spits out 29.97 fps interlaced, and the following ChangeFPS(23.976) will probably ruin the output (because now IVTC is required).

[videoh]

Seems to me you need to fix your scripts. You can make a simple batch file to do it, yes? While you are at it, define everything and do not rely on defaults. Not meaning to be contrary, just pragmatic and realistic. I think it's a no-brainer to honor pulldown by default.

[manolito]

So far your source filters have been the only ones which do it that way. Everybody else (ffms2, LSMASH and DirectShowSource based filters) did it differently, and I kind of got used to it. That does not mean that I do not see the merits or your approach, but over the years I have established workflows which are mainly geared at other source filters than yours.

And at my age you do not change your old ways easily. Especially since I use ffms2 much more often than LSMASH, and so far ffms2 does not honor pulldown flags by default.

[videoh]

So now there are two important source filter authors that think the intent of the stream should be honored. There are many cases where irregular pulldown, hybrid content, etc., preclude an automatic rate conversion, such as you suggest, as it would destroy AV sync.

Keep the inner child alive. Like Sherman!

[manolito]

There are many cases where irregular pulldown, hybrid content, etc., preclude an automatic rate conversion, such as you suggest, as it would destroy AV sync.


Since I live in PAL land, I have never encountered such clips with irregular pulldown or hybrid content. Can anybody provide a link to such a clip?

[kedautinh12]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200118/L-SMASH-Works_20200118.7z


VideoSource: Fix missing frames in some H264 streams.
LWLibav: Change the default of 'repeat' to true.
LWLibav: Fix VP8 decoding issue with alt-ref frames.


Can you update support vfr to cfr like ffms2???

[manolito]

Can you update support vfr to cfr like ffms2???

I do not understand your problem... :confused:

ffms2 and LSMASH both support VFR to CFR conversion, and they use identical methods. As soon as the user specifies FPSNUM and FPSDEN in the parameter list, both tools will spit out a CFR clip. They do this by duplicating or dropping frames as necessary to reach the specified frame rate.

So what exactly is your problem with LSMASH?

[kedautinh12]

I do not understand your problem... :confused:

ffms2 and LSMASH both support VFR to CFR conversion, and they use identical methods. As soon as the user specifies FPSNUM and FPSDEN in the parameter list, both tools will spit out a CFR clip. They do this by duplicating or dropping frames as necessary to reach the specified frame rate.

So what exactly is your problem with LSMASH?

I changed VFR to CFR with FPSNUM and FPSDEN but video after encoded video don't sync with audio. But it's work with directshowsource mod

my script:
LSMASH: LWLibavVideoSource("C:\Users\84945\Downloads\Otome Shinto - Otome Shinto no Uta [1440x1080 h264 M-ON! HD].ts", fpsnum=30000, fpsden=1001)

and directshowsource mod: dss2("C:\Users\84945\Downloads\Otome Shinto - Otome Shinto no Uta [1440x1080 h264 M-ON! HD].ts", fps=29.970).AssumeFPS(30000,1001)
ConvertBits(8)

video test: https://drive.google.com/open?id=1wUKzCzij8HJl22ElhD8xMPkrUHv9p3OT

[sneaker_ger]

my script:
LSMASH: LWLibavVideoSource("C:\Users\84945\Downloads\Otome Shinto - Otome Shinto no Uta [1440x1080 h264 M-ON! HD].ts", fpsnum=30000, fpsden=1001)
Where is the audio in your script?

[kedautinh12]

Where is the audio in your script?

I use megui and use auto encode so don't need audio script

[sneaker_ger]

Sync is relation video to audio. You only provide us with info about video. How are we supposed to reproduce your test?

(Actually for me your sample is only a black screen with the "M O N" logo at the bottom right, just a small blink of the real video at the start.)

[kedautinh12]

Sync is relation video to audio. You only provide us with info about video. How are we supposed to reproduce your test?

(Actually for me your sample is only a black screen with the "M O N" logo at the bottom right, just a small blink of the real video at the start.)

I fixed with trim from frame 31 to end and cut audio like this with function avs cutter of megui but error vfr to cfr. When i used with directshowsource mod don't need trim and vfr to cfr work fine

[manolito]

I played with the uploaded sample a little bit. Just playing it in DirectShow based players works fine, but after converting it I get the same problem like sneaker_ger (just a small blink of the video at the start). So your sample seems to have problems to begin with...

Another likely cause for issues is the format. Interlaced HD transport streams are always a challenge for source filters, so I made it a habit to repack such sources into an MKV container before trying to convert them.

But the real problem about audio sync does come from the source filter. Captured transport streams often have a huge audio delay. Your sample has an audio delay of -632 ms, after repacking to MKV the delay is -631 ms (almost identical). And different source filters treat such delays differently.

For DirectShow based souce filters as well as for ffms2 my experience is that this delay has to be discarded when muxing converted audio and video streams. LSMASH hehaves differently, the delay needs to be honored for perfect audio sync. (I believe that the DG source filters behave identically). I think that the reason is that some source filters just delete video frames at the start before a valid key frame while others like LSMASH repeat the first valid video frame up to the point where the audio starts.

On my tests I got perfect audio sync with the uploaded sample by discarding the delay value under DSS2Mod and honoring the delay value for LSMASH.

[kedautinh12]

I played with the uploaded sample a little bit. Just playing it in DirectShow based players works fine, but after converting it I get the same problem like sneaker_ger (just a small blink of the video at the start). So your sample seems to have problems to begin with...

Another likely cause for issues is the format. Interlaced HD transport streams are always a challenge for source filters, so I made it a habit to repack such sources into an MKV container before trying to convert them.

But the real problem about audio sync does come from the source filter. Captured transport streams often have a huge audio delay. Your sample has an audio delay of -632 ms, after repacking to MKV the delay is -631 ms (almost identical). And different source filters treat such delays differently.

For DirectShow based souce filters as well as for ffms2 my experience is that this delay has to be discarded when muxing converted audio and video streams. LSMASH hehaves differently, the delay needs to be honored for perfect audio sync. (I believe that the DG source filters behave identically). I think that the reason is that some source filters just delete video frames at the start before a valid key frame while others like LSMASH repeat the first valid video frame up to the point where the audio starts.

On my tests I got perfect audio sync with the uploaded sample by discarding the delay value under DSS2Mod and honoring the delay value for LSMASH.

I think error don't relate with delay of audio, i used aegisub and make karaoke sub sync audio and video and after encoded only audio sync with karaoke sub but video

[kedautinh12]

I am wondering how did you determine the sync issue was caused by VFR->CFR conversion. Have you tried omitting the fpsnum and fpsden arguments? And was there even VFR->CFR conversion taking place? It seems a no-op to me since the frame rate of your sample is already 29.97 (30000/1001) fps.

Why i use vfr to cfr same fps in directshowsource mod (value fps in script of directshowsource mod are autoset by megui) and it's work perfectly??

[manolito]

Since I live in PAL land, I have never encountered such clips with irregular pulldown or hybrid content. Can anybody provide a link to such a clip?

Bump...

[hello_hello]

Does the repeat option change the way the video is decoded in respect to chroma subsampling? I'm just wondering what happens when it's set to true and a source is progressive? I've never really understood how chroma subsampling is decoded in that respect.

repeat
Reconstruct frames by the flags specified in video stream and then treat all frames as interlaced if set to true and usable.

Is it safe to assume the ability to use index files as the source will remain broken?

[hello_hello]

So now there are two important source filter authors that think the intent of the stream should be honored. There are many cases where irregular pulldown, hybrid content, etc., preclude an automatic rate conversion, such as you suggest, as it would destroy AV sync.

Would it destroy the AV sync? The decoder just drops or duplicates frames for a constant frame rate.

If you have a DVD that's a mixture of soft telecined 23.976fps and 29.97fps progressive or interlaced, and repeat is false but you use FPSNum and FPSDen to convert to a constant frame rate of 29.97fps, you should end up with the progressive sections having every 4th frame repeated and the audio sync should be intact, unless I'm missing something. I'm not saying it's ideal, but unless you want a VFR output, it needs to be one or the other. It's only if you don't enable repeat flags or do a VFR conversion that you'll end up with audio sync problems.

The help file says the repeat option is ignored when VFR to CFR conversion is specified, which might be why it's always been false by default, although I'm not sure how often you'd want a CFR to VFR conversion and honour repeat flags at the same time.

ffms2 can write a timecodes file as it indexes, so in that respect, not honouring repeat flags by default probably makes sense as you can use them for a VFR encode. There'd be no need to honour repeat flags and then use TIVTC to run an analysis pass to return the frame rate to variable.

I don't recall lsmash having an option to write a timecodes file though, so repeat being true by default probably makes sense.

[videoh]

Bump... Here is one:

http://rationalqm.us/misc/lainvob.vob

It's a small sample provided under fair use. I have some larger off-air TS captures that are similar. This stuff is ubiquitous in NTSC land.

[videoh]

It's only if you don't enable repeat flags ... that you'll end up with audio sync problems. Correct, and that's the point because manolito wants to ignore pulldown by default (he objects to honoring pulldown by default because it breaks his existing scripts).

[manolito]

Here is one:

http://rationalqm.us/misc/lainvob.vob

It's a small sample provided under fair use. I have some larger off-air TS captures that are similar. This stuff is ubiquitous in NTSC land.

Thanks for the clip, but this one does not do anything for me...

I know that you are a "pedal to the metal" guy, and I prefer encoder GUIs.

First of all the sample clip is not suitable to judge A/V sync. And then most encoder GUIs use MediaInfo to determine the source properties, and this clip appears as 29.97 interlaced TFF. No pulldown detected. And the encoder treats the clip exactly like this.

I used AVStoDVD to compare conversions first using DGIndex and then using ffms2. Both looked identical to me.

To get meaningful results the source clip needs some scenes where I can judge A/V sync, and then it would help if MediaInfo would detect 3:2 Pulldown. If MediaInfo just detects 29.97 pure interlaced then there is no way that the encoder GUI could do any fancy things.


So it all comes down to the philosophy you apply. I want things to work universally under an encoder GUI like AVStoDVD or StaxRip. You are able to do all this stuff manually and find the best way to encode such clips, but this is something most users cannot (and do not want to) do.

[videoh]

The clip has irregular pulldown (which is what you asked for). If you want a longer sample, buy the DVD.

I want things to work universally... I want a unicorn for Christmas. ;)

And, hey, I'm not using any super powers here. A quick preview in DGIndex clearly shows the irregular pulldown. Press F5 and let it play to the end. Then look at the field repeats in the info dialog and the film percentage in the DGI file.

If you have a clip with hybrid soft and hard pulldown, you can only honor pulldown and then do IVTC with a filter. Ignoring the pulldown would ruin AV sync. All of this is well known in NTSC land.

[manolito]

A quick preview in DGIndex clearly shows the irregular pulldown. Press F5 and let it play to the end. Then look at the repeat fields and film percent in the info dialog.


No "Average Joe" user would ever want to do this. You are from Pluto and I am from Mars...

[kedautinh12]

Repeat
Reconstruct frames by the flags specified in video stream and then treat all frames as interlaced if set to true and usable.

I test many raw with repeat and still interlaced in some frame, i think it's only use with raw have duration of video don't sync with duration of audio

[videoh]

No "Average Joe" user would ever want to do this. You are from Pluto and I am from Mars... Average Joes also do stuff like deinterlace 3:2 pulldown, and resize before deinterlacing. This is Doom9, where things are done right! Maybe VideoHelp is more your style?

[videoh]

I test many raw with repeat and still interlaced in some frame, i think it's only use with raw have duration of video don't sync with duration of audio What do you mean by "raw"? Of course, 3:2 pulldown is going to appear to have interlaced frames. So I don't get your point.

[kedautinh12]

What do you mean by "raw"? Of course, 3:2 pulldown is going to appear to have interlaced frames. So I don't get your point.

Oh, sorry i call .ts file is raw, my mistake

[manolito]

This is Doom9, where things are done right! Maybe VideoHelp is more your style?

Being your old arrogant self again...:p

[videoh]

Call it confidence born of deep theoretical understanding and a broad base of experience.

[hello_hello]

The clip seems to be a mixture of soft and hard telecine and interlaced/progressive. I tried 4 different decoding/filtering methods, all using ffms2 as I wanted to try a sample using the timecodes file it (optionally) creates when indexing.
RFFMode=0 is the same as Repeat=false for lsmash.

1 VFR ffms2 timecodes - 1762 frames.mkv
The hard telecine parts are field matched and left at 29.97fps. Average frame rate 27.583fps.
FFVideoSource("E:\lainvob.mkv", Threads=1, TimeCodes="FFMSTimecodes.txt", RFFMode=0)
TFM()

2 VFR TIVTC 2 pass - 1662 frames.mkv
Same as #1 except the hard telecine parts are decimated to 23.976fps. Average frame rate 26.028fps.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=1)
TFM(Input="D:\TFM.txt")
TDecimate(Mode=5, Hybrid=2, Input="D:\TDec.txt", tfmin="D:\TFM.txt", mkvout="D:\Times.txt")

3 CFR conversion ffms2 - 1916 frames.mkv
The hard telecine parts are field matched and left at 29.97fps. The soft telecine parts would have every 4th frame duplicated.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=0, FPSNum=30000, FPSDen=1001)
TFM()

4 CFR conversion TIVTC frame blending - 1916 frames.mkv
The hard telecine parts are field matched and decimated, and the soft/hard telecined parts are frame blended to 29.97fps.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=1)
TFM()
TDecimate(Mode=1, Hybrid=3)

All cropped and resized to 640x480.
encodes.zip (https://www.sendspace.com/file/mriedq) (72.6 MB)

And one more at the average frame rate of 27.568fps (1762 frames). The A/V sync is definitely different. At roughly the 41 second point she should be mouthing the word "much" in sync with the audio, as happens with all the previous samples.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=0)
TFM()
Average frame rate.mkv (https://www.sendspace.com/file/fqrl6q)

manolito,
for a method that always works with encoder GUI's wouldn't it be better to always honour repeat flags as DGIndex does by default? I don't understand the logic behind not doing so unless you want a VFR output, or there's not much soft telecined content and the repeated frames wouldn't be too noticeable. Admittedly PAL is generally much simpler to deal with, but for NTSC you can always tell if there's soft pulldown by disabling repeat flags and using Info() to check the frame rate.

[hello_hello]

repeat
Reconstruct frames by the flags specified in video stream and then treat all frames as interlaced if set to true and usable.

I'm still not clear on whether the repeat flag effects the chroma upsampling (interlaced vs progressive) or am I looking at it all wrong?

[manolito]

First of all I really do not want to do a thorough analysis of such clips using DGIndex. And I absolutely want to avoid VFR, all my conversions need to come out as CFR.

The 2 GUIs I use are AVStoDVD for DVD output and StaxRip 32-bit for other output formats (I also use DVDStyler and dmMediaConverter, but they do not count here because they do no use AviSynth). And both GUIs employ MediaInfo to detect the source properties.

The clip which was uploaded by our friend is detected as 29.97 fps Interlaced TFF. No soft pulldown is detected. MediaInfo only reports 2:3 pulldown if the clip has the standard regular pulldown pattern for 23.976 to 29.97 telecining.

Both GUIs ALWAYS do VFR to CFR conversion right at the source filter level. If the source aready is CFR then this operation is useless, but it won't do any harm either.

The uploaded clip will be treated like this:
DGIndex / DGDecode:
The default "Honor Pulldown Flags" is used, followed by a ChangeFPS(29.97). Then users can either leave it interlaced (the default) or deinterlace the clip. IVTC is not offered because MediaInfo did not report pulldown.

DSS2Mod (the same for DirectShowSource):
DSS2Mod does not honor pulldown flags. The uploaded clip will have the "fps=29.97" parameter in the DSS2Mod call, so there will be lots of dupes. Then again you can leave it interlaced or use a deinterlacer.

ffms2 and previous LSMASH builds:
Pretty much the same as DSS2Mod.

This is not optimal, but it is very watchable, and I never saw any A/V sync problems.


If the source has regular 2:3 pulldown then it gets a little different. MediaInfo will report 23.976 progressive and 2:3 pulldown. When using DGIndex with the default Honor Pulldown Flags then AVStoDVD will force an IVTC right after the source filter. When DSS2Mod (or ffms2 or previous LSMASH) is used then the source filter will force VFR to CFR conversion to 23.976 fps. There will be dropped or duplicated frames, but again no A/V sync problems, and the output will be pure progressive.

[manolito]

for a method that always works with encoder GUI's wouldn't it be better to always honour repeat flags as DGIndex does by default?

Main reason is that DSS2Mod is my standard source filter (AVSToDVD uses it by default), and all DirectShow based source filters cannot honor pulldown flags.

I found this out the hard way because it is not documented anywhere.

If your source is a regular soft telecined NTSC Film clip and you open it with DSS2Mod (or DirectShowSource) without the "fps=" parameter then the output will be 29.97 fps. For some time I thought that DSS2Mod did indeed honor the pulldown flags, but I was quite wrong. The output had no pulldown pattern, it was purely progressive with lots of dupes. Which means that DSS2Mod decoded the clip to 23.976 first and then added dupes to bring the rate up to 29.97.

I still think that this is a stupid behavior, so for sources with 2:3 pulldown I always specify "fps=23.976" in the DSS2Mod parameter list.

[hello_hello]

Both GUIs ALWAYS do VFR to CFR conversion right at the source filter level. If the source aready is CFR then this operation is useless, but it won't do any harm either.

It can. I haven't tested with vob files, but I've seen the problem with both ffms2 and lsmash for AVI and MKV sources. I think it's something to do with jitter in the source timecodes, but it's possible that when the frame rate is constant and you tell ffms2 or lsmash to convert to that constant frame rate, they'll drop and repeat frames in places when in theory they should do nothing. As a result I never use frame rate conversion unless I want to convert the frame rate.
https://forum.doom9.org/showpost.php?p=1703007&postcount=4395

Anyway... there's definitely soft pulldown in that clip, unless I'm interpreting it wrong. Just not at the beginning, which is probably where MediaInfo looks.

If you remux the sample as an MKV and extract the timecodes, you'll see they're 33ms apart at the beginning, which means either progressive, interlaced, or hard pulldown, all at 29.97fps.

If you scroll down to frame number 652, the timecodes look like this:

22039
22089
22122
22172
22206
22256
22289
22339
22372
22422
22456
22506

The only explanation I can think of for the timecodes being 50ms apart is due to the repeat flags not being included in the timecodes.

1000 / (30000 / 1001) = 33.37 ms per frame.
1.5 frames = 50.05 ms

But does that mean if you don't honour pulldown flags, instead of telecined frames at 33.37 ms per frame, or progressive frames at 41.7ms per frame (23.976fps) after decimation, you're getting frames alternating between 50ms and 33ms for an average of 41.5ms (roughly)?
I hadn't thought about it like that until now. Mind if you do a VFR to CFR conversion with DSS or Repeat=true etc, it'd probably even them out to 33.37ms durations, with every forth frame repeated, or to an even 41.7ms per frame if you convert to 23.976fps.

[edcrfv94]

Vapoursynth R45 not support?

[manolito]

It can. I haven't tested with vob files, but I've seen the problem with both ffms2 and lsmash for AVI and MKV sources. I think it's something to do with jitter in the source timecodes, but it's possible that when the frame rate is constant and you tell ffms2 or lsmash to convert to that constant frame rate, they'll drop and repeat frames in places when in theory they should do nothing. As a result I never use frame rate conversion unless I want to convert the frame rate.
https://forum.doom9.org/showpost.php?p=1703007&postcount=4395


If this is true then I would call it a bug in ffms2 and lsmash.

In most of my scripts with ffms2 and lsmash I do not even use the fpsnum and fpsden parameters, I use ChangeFPS(..) right after the source filter call. This method solves 2 problems in 1 step:
First of all it does the VFR to CFR conversion (if applicable), and secondly the ChangeFPS command ensures linear access to the source filter. This is necessary for AVS+ in MT mode, because without linear access the speed of the source filter will drop to a snail's pace.

[hello_hello]

If this is true then I would call it a bug in ffms2 and lsmash.

It was a problem reported by a couple of other people too, after the ffms2 CFR to VFR conversion was automatically added to scripts by MeGUI, and I've seen lsmash do the same. That was actually my idea, as I thought it'd solve the problem of not being able to use MeGUI to convert VFR sources automatically, but then the CFR source problem appeared, plus it relied on MeGUI getting the new constant frame rate right, which should always be 29.97fps for NTSC-type sources, or at least the same as the maximum frame rate. These day's MeGUI's OneClick encoder can extract the timecodes from MKVs and use them for a VFR encode, so there's no need for a messy constant frame rate conversion. For DVD encoding I understand a constant frame rate is necessary, but when the destination is MKV I rarely contemplate a VFR to CFR conversion rather than a VFR output.

In most of my scripts with ffms2 and lsmash I do not even use the fpsnum and fpsden parameters, I use ChangeFPS(..) right after the source filter call. This method solves 2 problems in 1 step:
First of all it does the VFR to CFR conversion (if applicable), and secondly the ChangeFPS command ensures linear access to the source filter. This is necessary for AVS+ in MT mode, because without linear access the speed of the source filter will drop to a snail's pace.

I'm experiencing déjà vu. Have we had this conversation before?
I don't understand how ChangeFPS can convert VFR sources to a constant frame rate. Say you have a source that's a mixture of film (soft telecine) and video and it's 50-50, so the average frame rate is 26.938fps. Without frame rate conversion or honouring repeat flags, that's how ffms2 and lsmash decode it. The film sections would play too quickly and the video sections too slowly, and the A/V sync would be lost. If you use ChangeFPS to change the frame rate to 29.97fps (for example), it'll add frames to both the film and video sections to bring the frame rate up to 29.97fps, but they'll both still play at the wrong speed. The film sections would have less frames added than if they were being decoded at 23.976fps and the video sections would have frames added that wouldn't be if they were decoded at 29.97fps. An ffms2 or lsmash frame rate conversion at least ensures the extra frames are added where they're supposed to be, because they're aware of how the source frame rate varies. ChangeFPS isn't.

I kind of get how ChangeFPS could ensure more linear frame requests from the source filter, but have you tried RequestLinear() from the TIVTC package? I think by default it requests frames 50 at a time from the upstream filter and caches 10 frames itself, although both can be adjusted.

[TheFluff]

But the real problem about audio sync does come from the source filter. Captured transport streams often have a huge audio delay. Your sample has an audio delay of -632 ms, after repacking to MKV the delay is -631 ms (almost identical). And different source filters treat such delays differently.

For DirectShow based souce filters as well as for ffms2 my experience is that this delay has to be discarded when muxing converted audio and video streams. LSMASH hehaves differently, the delay needs to be honored for perfect audio sync. (I believe that the DG source filters behave identically). I think that the reason is that some source filters just delete video frames at the start before a valid key frame while others like LSMASH repeat the first valid video frame up to the point where the audio starts.

I doubt your theory regarding incomplete GOP's is correct. In the FFMS2 case, if you're also decoding audio with it, what you're seeing is probably just the default audiodelay setting (https://github.com/FFMS/ffms2/blob/master/doc/ffms2-avisynth.md#int-adjustdelay---1). That is, if you have a -631ms delay in the container, that means (in practical terms) that the audio stream should start playback 631ms before the video stream. With the default audiodelay setting, FFMS2 will then discard the first 631ms of the audio stream in order to make video and audio start at the same time. Since Avisynth doesn't support timecodes for neither audio nor video, there's no way for it to know that one stream starts before another, so cutting off the audio or filling it in with silence is the only way to adjust sync. Other source filters probably just default to giving you the audio as it is.


I don't understand how ChangeFPS can convert VFR sources to a constant frame rate.

You are correct, it can't (because it lacks access to the timecodes). If the source was actually VFR, it would indeed cause desync.


As far as the discussion regarding honoring repeat field flags/soft pulldown or not, I basically think both sides are wrong. Honoring RFF's only really makes sense if you intend for the output to be interlaced, or if you're intending to deinterlace the entire stream because it's mostly interlaced content anyway and you can't be bothered to treat it as hybrid. In any other situation you're just doing a lot of pointless work by adding fields you're going to remove later anyway. Additionally, RFF's are a way of saying "look, this is actually progressive, but we have to pretend it's interlaced for compatibility reasons", so there's no reason to honor them if you don't actually want interlacing in the first place.

However, in the context of Avisynth specifically, not honoring RFF's is kind of a pain in the ass. Avisynth doesn't have any per-frame metadata, not even timecodes, so there's no builtin way of signalling which frames are interlaced and which aren't, nor any way of dealing with VFR, so dealing with the hybrid content that disregarding RFF's is likely to result in requires significantly more manual effort than it does in e.g. Vapoursynth. So in that context honoring RFF's does make life easier.

[manolito]

So far everybody has been telling me that my way of dealing with NTSC clips with irregular pulldown or hybrid content will inevitably lead to A/V sync problems.

Then I asked (repeatedly) for a link to such a clip which would prove it. All I got was a link to a clip which is absolutely useless to judge A/V sync.

Please let me ask for it again:
I need a clip taken from a DVD which will show that my way of dealing with such clips is flawed. And which will hopefully allow me to find a way to deal with it without using DGIndex.

[TheFluff]

I haven't had a DVD drive in my computer for years now and I've deleted most of the old stuff that could be interesting, so I can't help you there. The clip would have to be fairly long too to make audio desync really noticeable, a few minutes at least.

[videoh]

Suppose a 29.97 stream's first half is video with no pulldown and the second half is pure soft 3:2 pulldown. If I understand correctly, the suggestion is to ignore pulldown and then apply ChangeFPS. But what does this really mean? You're going to have to specify some decimated frame rate, but you can't just assume 23.976 because that would be correct only for the second half. And we don't want any ChangeFPS action on the first half anyway. For totally irregular pulldown it could be any old crazy rate you would have to choose correctly if ChangeFPS is going to be any good.

I've also seen progressive transport streams with bursts of soft frame repeats in an irregular manner (presumably for compression of black or static video). How will your way deal with those? You have to honor the frame repeats.

my way of dealing with such clips

I would welcome a more detailed explanation of your proposed handling for hybrid and irregular pulldown. Sure it could be reasonable for pure soft 3:2 pulldown, but that's not what we are talking about. Can you give your exact script for handling lainvob.vob your way?

Sure, implementing pulldown correctly and doing random access for it in a source filter is challenging and difficult, but that's not a good reason to do the wrong thing.

[hello_hello]

Then I asked (repeatedly) for a link to such a clip which would prove it. All I got was a link to a clip which is absolutely useless to judge A/V sync.

Please let me ask for it again:
I need a clip taken from a DVD which will show that my way of dealing with such clips is flawed. And which will hopefully allow me to find a way to deal with it without using DGIndex.

The clip videoh uploaded will lose AV sync if it's not dealt with correctly. The spot it's easiest to see is around the 41 second point where she mouths the word "much" to the sung vocal.
I don't know whether you've looked at the sample encodes I uploaded here (https://forum.doom9.org/showthread.php?p=1896725#post1896725), but the A/V sync is okay for all the samples in the zip file, but not the separate average frame rate encode.

Another way to check the A/V sync is to open the original sample and an encoded version in different instances of MPC-HC and play them together. I have MPC-HC configured so I can pause and resume playback quickly by clicking on the video (I disable double clicking switching to full screen and use a middle click for that instead, because a double click for fullscreen is mental anyway). When they're running, it's then easy to quickly pause and resume playback for one instance of MPC-HC until the audio of both are in sync. Generally after a few fast double clicks on each instance of the player as required, you can sync the audio well enough to hear it start to phase cancel, rather than hearing a delay between them. With the audio in sync, the video should be too.
I sometimes use that method to check the A/V sync for CFR encodes (compared to the source to make sure any audio delay is correct when I suspect it's not). If the audio is phasing and the scene changes happen at exactly the same time in both players, you know the A/V sync is the same.

PS. If you try the MPC-HC method it's best to set "left down" for the play/pause option. If I remember correctly, "left up" lets you drag the player around by clicking on the video and holding the left mouse button down, and it can interfere with the ability to quickly pause and resume playback by clicking on the video. For "left down", the player doesn't have to try to guess what you're wanting to do when you click on the video.

[videoh]

Thank you, hello_hello. Seems I have hit the theoretical part and you have hit the empirical part. Gonna be tough to beat!

[hello_hello]

But the real problem about audio sync does come from the source filter. Captured transport streams often have a huge audio delay. Your sample has an audio delay of -632 ms, after repacking to MKV the delay is -631 ms (almost identical). And different source filters treat such delays differently.

For DirectShow based souce filters as well as for ffms2 my experience is that this delay has to be discarded when muxing converted audio and video streams. LSMASH hehaves differently, the delay needs to be honored for perfect audio sync. (I believe that the DG source filters behave identically). I think that the reason is that some source filters just delete video frames at the start before a valid key frame while others like LSMASH repeat the first valid video frame up to the point where the audio starts.

I only just looked at the sample, but as soon as I see a negative audio delay in an MKV I check to see what's required to remove it. In the case of that sample, the video stream has a positive video delay of 631ms which MediaInfo reports as a -631ms audio delay. The simple fix is to remux it while applying a -631ms delay to every stream. You'll be left with either a zero or a small positive audio delay after MKVToolNix cuts the beginning of the audio.

If you open that sample with gMKVExtractGUI you'll see what's happening. If gMKVExtractGUI extracts the audio, it'll write a -631 delay to the extracted stream under the assumption the 631ms video delay will be lost during re-encoding. Most extraction programs would just write a zero delay. If you were to remux it with the original video while applying the -631ms delay, which would be a natural assumption, the A/V sync will change.
Writing the audio delay relative to the video when extracting the audio was actually my idea, after realising a video delay isn't all that uncommon.
The author of MKVCleaver kindly took it one step further so MKVCleaver can be configured to write both audio delays to the audio stream. The delay relative to the video and also the container audio delay. MediaInfo reporting a negative audio delay is a strong indication there's a positive video delay in an MKV, because for MKV there's no such thing as negative delays, but it's possible for the audio delay to be positive even when there's a video delay, if the audio delay is larger.

Anyway, as gMKVExtractGUI shows you both the stream delays and the video/audio delays relative to each other, it's easy to work out what's going on. I've no idea how the different source filters handle that sort of thing when decoding the audio in the original container. They should start decoding where the video starts, but remuxing the sample as an MKV while applying a -631ms delay to both the audio and video will remove that variable. I use the original audio most of the time rather than re-encode it, or if I do encode it I extract it first anyway, so it's more important to know the relative delay than the container delay.

The first delay for each stream is the container delay. The second delay for the audio stream shows their relative delays. The second delay for the video stream is the relative delay minus any audio container delay, I think. That one confuses me.

Remuxed

https://i.postimg.cc/TYCZS34m/01.gif

Remuxed while applying a -131ms delay to both streams

https://i.postimg.cc/RZhjzxkY/02.gif

Remuxed while applying a -631ms delay to both streams

https://i.postimg.cc/7YxFm2mw/03.gif

Here's what the help files say...

FFMS2:
int adjustdelay = -1
Controls how audio delay is handled, i.e. what happens if the first audio sample in the file doesn't have a timestamp of zero.
The following arguments are valid:

- **-3**: No adjustment is made; the first decodable audio sample becomes the first sample in the output.
- **-2**: Samples are created (with silence) or discarded so that sample 0 in the decoded audio starts at time zero.
- **-1**: Samples are created (with silence) or discarded so that sample 0 in the decoded audio starts at the same time as frame 0 of the first video track. This is the default, and probably what most people want.
- **Any integer >= 0**: Same as -1, but adjust relative to the video track with the given track number instead.
If the provided track number isn't a video track, an error is raised.
-2 obviously does the same thing as -1 if the first video frame of the first video track starts at time zero.
In some containers this will always be the case, in others (most notably 188-byte MPEG TS) it will almost never happen.

Lsmash
av_sync (default : false)
Try Audio/Visual synchronization at the first video frame of the video stream activated in the index file if set to true.

[manolito]

Yes, this is what I found out by trial and error a while ago when DVB-T2 was introduced in Germany (HEVC video, E-AC3 or AAC-LATM audio). The captured transport streams always have this huge audio delay, and for DSS2Mod and ffms2 these delays have to be ignored, for LSMASH the delays must be corrected.

So this is all caused by different default behavior of these source filters. Why in the world can the relevant source filter authors not agree on a uniform behavior?

The problem is that I never use my source filters manually, they are called by a GUI where the call mostly is hard-coded in the GUI. Either users cannot edit this call at all, or it requires that the user edits the generated AVS script.

This is not an option for most users. These GUIs are geared at average users, not at video specialists like you and DG. Since some of these GUIs are no longer maintained, it may be possible to use newer helper applications (like AVS filters or encoders), but only if the newer versions stick with their original defaults. This is the main reason why I always hate when the author of a tool which is widely used by other software suddenly decides that he wants to change defaults or calling conventions without any regard for backwards compatibility. MediaInfo is a good example, most GUIs which use it stopped working with versions after 18.5. And now the same thing happens with the latest LSMASH build by HolyWu. I really hate this, I call it "Apple Attitude".

[Myrsloik]

...

So this is all caused by different default behavior of these source filters. Why in the world can the relevant source filter authors not agree on a uniform behavior?

...


The source files are incorrectly flagged so often anything automatic will fail. We encoders are all so elitist because WE KNOW BETTER THAN THE MACHINES!

[videoh]

@manolito

Are you going to give us your script for lainvob.vob that does things "your way"?

[manolito]

Oh sure, I thought that you had noticed my reply here:
https://forum.doom9.org/showthread.php?p=1896679#post1896679

MediaInfo reports this:
Frame rate : 29.970 (30000/1001) fps
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First

And since no pulldown flags are reported, AVStoDVD treats it like a 29.97 interlaced TFF clip. The AVS script generated by AVStoDVD looks like this:
LoadCPlugin("E:\Programme\AVStoDVD\Lib\ffms2.dll")
Audio = FFAudioSource("F:\Download\lainvob.vob", track=-1)
Video = FFVideoSource("F:\Download\lainvob.vob", track=-1, fpsnum=2997, fpsden=100, colorspace="YV12", seekmode=2)
Video = Video.ConvertToYV12(interlaced=true)
AudioDub(Video, Audio)

ffms2 does not honor pulldown flags, so the fpsnum and fpsden parameters insert duplicates to bring up the frame rate to 29.97 fps. The resulting clip looks every bit as good as the original to me, and I cannot detect any A/V sync problems (within the limits of the clip characteristics).

The converted clip is here:
https://www.sendspace.com/file/0zcoxv

[hello_hello]

And since no pulldown flags are reported, AVStoDVD treats it like a 29.97 interlaced TFF clip.

Just for the sake of arguing, doesn't "treating it like a 29.97 interlaced TFF clip" and telling ffms2 to do a VFR to 29.97fps CFR conversion seem a little contradictory? If it's a 29.97 interlaced TFF clip there's no need to a VFR to CRF conversion. :)

ffms2 does not honor pulldown flags, so the fpsnum and fpsden parameters insert duplicates to bring up the frame rate to 29.97 fps. The resulting clip looks every bit as good as the original to me, and I cannot detect any A/V sync problems (within the limits of the clip characteristics).

Yeah, there's no reason why that shouldn't be in sync.
Because the sample is animation you get get away with lots of duplicate frames and not see a difference. If it was live action though, with lots of film sections, every fourth frame being repeated would probably get annoying fairly quickly.

I wonder why a program dedicated to DVD encoding wouldn't tell ffms2 to honour the pulldown flags by default, especially for a 29.97fps source? The script seems to indicate it's encoding the video as interlaced anyway..... although now I'm thinking about it, I recall the ffms2 help file says it'll throw an error if honouring repeat flags is enabled but the stream doesn't have any. I can't say I've tested that, but I have no idea what the logic behind that behaviour might be.

[hello_hello]

This is the main reason why I always hate when the author of a tool which is widely used by other software suddenly decides that he wants to change defaults or calling conventions without any regard for backwards compatibility. MediaInfo is a good example, most GUIs which use it stopped working with versions after 18.5. And now the same thing happens with the latest LSMASH build by HolyWu. I really hate this, I call it "Apple Attitude".

I'm still waiting to learn why one of the methods I suggested for determining when lsmash writes the source path to the index file wouldn't keep GUI's that use them as the source happy while not inconveniencing those who like to move and rename their source/index files.

[TheFluff]

I recall the ffms2 help file says it'll throw an error if honouring repeat flags is enabled but the stream doesn't have any. I can't say I've tested that, but I have no idea what the logic behind that behaviour might be.
That line is not strictly accurate, I think. AFAIK, you'll only get an error if you try to apply RFF's to a video stream encoded using a codec that's incapable of supporting them at all.

[videoh]

Thank you, manolito, for your script. It's easy to show that it is producing garbage compared to a proper conversion. I'll just look at video here (although audio sync also fluctuates significantly). Your script is producing 29.97 when most people will want 23.976 without dupes. Nevertheless, I omit the TDecimate call in my version so as to be able to compare the methods. Also note that I do not deinterlace either in DGSource() or via TFM() postprocessing, so the DG method is deriving good progressive frames throughout. Here is the script I use for comparing the methods:

LoadCPlugin("ffms2.dll")
manolito = FFVideoSource("lainvob.vob", track=-1, fpsnum=2997, fpsden=100, colorspace="YV12", seekmode=2).subtitle("manolito")

loadplugin("d:\don\Programming\C++\dgdecnv\DGDecodeNV\x32\release\dgdecodenv.dll")
loadplugin("D:\Don\Programming\C++\Avisynth filters\tritical stuff\TIVTC\tivtc.dll")
dgsource("G:\Streams\MPG\Pulldown\lainvob.dgi",fieldop=0)
dg=tfm(PP=0).subtitle("DG")
#tdecimate()

stackhorizontal(dg,manolito)

Now load the script in VirtualDub and step linearly from the beginning. There will be lots of laced frames seen in the manolito version. Here are some notable locations:

548
606
714
730-840 birds are interlaced
876 lacing on scene change
900 lacing on scene change
933- lots of laced frames from here

Don't do random access because ffms2 doesn't do it correctly and my numbers above are for linear play from the beginning.

Any reasonable person would reject the manolito method. It's way worse than "not optimal". As several have claimed, the correct way to handle material like this is to honor the pulldown and then do IVTC using (for example) TFM/TDecimate.

[manolito]

Now load the script in VirtualDub and step linearly from the beginning. There will be lots of laced frames seen in the manolito version.

You don't get it. I DO NOT watch my movies by stepping through the frames. It is meaningless.

Of course my conversion has interlaced frames. When I make a DVD the common recommendation is to keep interlaced sources interlaced and let the TV handle the deinterlacing.

I showed the 2 versions of the clip to a friend without telling him what to look for (he is a journalist for a public TV broadcaster). He could not tell which version was which.

I think we can stop this here...

[videoh]

I think we can stop this here...
No way, Jose.

Of course my conversion has interlaced frames. Translation: Of course my method sucks.

DG derangement is a terrible thing. I will pray for you.

[TheFluff]

You don't get it. I DO NOT watch my movies by stepping through the frames. It is meaningless.

Of course my conversion has interlaced frames. When I make a DVD the common recommendation is to keep interlaced sources interlaced and let the TV handle the deinterlacing.

I showed the 2 versions of the clip to a friend without telling him what to look for (he is a journalist for a public TV broadcaster). He could not tell which version was which.

I think we can stop this here...

If you want interlaced output then why on earth would you not honor pulldown flags? :confused:

[videoh]

Echoing TheFluff, if you are "making a DVD" why is any conversion necessary at all? The lainvob.vob clip is already a DVD sample.

People want to make film-rate MKVs, etc. Your script fails badly for that. The progressive frames are waiting there, begging to be extracted.

a journalist for a public TV broadcaster That explains everything. Fake news!

[manolito]

If you want interlaced output then why on earth would you not honor pulldown flags? :confused:
I already said this repeatedly:

I use a GUI for my conversions. For DVD conversions I use AVStoDVD. AVStoDVD analyzes the source using MediaInfo. For the clip in question MediaInfo DOES NOT report pulldown flags, it reports 29.97 interlaced TFF instead. And if AVStoDVD thinks that there are no pulldown flags then it will NOT try to honor them.

To save the reputation of AVStoDVD I have to add that whenever it encounters an MPEG2 or VOB source it strongly suggests to index the source with DGIndex. And DGIndex does honor pulldown flags by default. I just have my reasons to avoid DG software as much as I can...

[videoh]

I just have my reasons to avoid DG software as much as I can... As I said, DG derangement is a terrible thing. Care to share those reasons with us?

Don't forget that we now have other source filters that do the right thing by honoring pulldown. I'm happy to know that DG tools have blazed the trail!

Did you have any comment on my earlier questions? For example, how would you handle a transport stream with irregular frame repeats?

[TheFluff]

Actually, I just realized that if you do this:
Video = FFVideoSource("F:\Download\lainvob.vob", track=-1, fpsnum=2997, fpsden=100, colorspace="YV12", seekmode=2)
What you're effectively doing is just applying pulldown. 60i sections will be left as they are, while 24p sections with RFF flags will get one of every 4 frames repeated, resulting in 30p. So you've effectively gotten the same result as honoring pulldown flags would have, except it's gonna look more stutter-y than normal soft pulldown would have, because instead of having the usual 3:2 cadence where every other frame gets an extra field, every fourth frame gets two extra fields instead. You put all of the catch-up you've accumulated over four frames in one place instead of spreading it out.

So yeah, don't do that. I still think ignoring pulldown flags is the best default and gives the best result in general if your goal is to encode progressive, but for your use case I don't see how it makes any sense.

e: Stephen got there first

[manolito]

Don't forget that we now have other source filters that do the right thing by honoring pulldown.
We have just one, and this is the latest HolyWu build of LSMASH. ffms2 and DSS2Mod are not following your lead so far.

Did you have any comment on my earlier questions? For example, how would you handle a transport stream with irregular frame repeats?
I did ask several times for a link to such a file. No result...

Care to share those reasons with us?
Bad Karma

[manolito]

What you're effectively doing is just applying pulldown.

Yes, I do know this. I repeat frames instead of fields, and most folks agree that repeating fields should give smoother motion. Being from PAL land I always found that even standard 3:2 NTSC pulldown results in ugly motion judder, it's just that people from NTSC land are so used to it that they don't even notice it any more.

[TheFluff]

I'd say what most people probably do around here is encode progressive content as progressive without duplicate frames, actually. Either by ignoring pulldown flags or by applying IVTC. They don't actually make interlaced displays anymore, y'know?

[Groucho2004]

They don't actually make interlaced displays anymore, y'know?Yes, but manolito is quite passionate about his good old CRT TV (as well as a non-SSE2 CPU). To each his own.

[manolito]

Groucho, you are absolutely correct. Keeping my old hardware I do hardly sacrifice anything, and I keep my carbon dioxide footprint low. Sustainability, repair instead of throwing things into the trash bin...

//OT//
I saw that you have compiled a ColorMatrix 2.6 build which I could use for UHD sources. It just does not like my non-SSE2 CPU. Any chance for a build which does not require SSE2?

It is not really too important because my ancient non-SSE2 computer will hardly see any UHD source files. But I like to have consistent tool versions on all of my machines.
//End OT//

[Groucho2004]

Any chance for a build which does not require SSE2?No problem.

[Richard1485]

Being from PAL land I always found that even standard 3:2 NTSC pulldown results in ugly motion judder, it's just that people from NTSC land are so used to it that they don't even notice it any more.

Yeah, but the judder caused by duplicating every fourth frame is noticeably worse than the result of 2:3 pulldown. Being accustomed to the latter doesn't make the former any less annoying.

[TheFluff]

Groucho, you are absolutely correct. Keeping my old hardware I do hardly sacrifice anything, and I keep my carbon dioxide footprint low. Sustainability, repair instead of throwing things into the trash bin...

I don't know much about CO2 emissions caused by the manufacture of TFT screens, but considering the German insistence on burning coal for power generation, the energy savings you would've had by now if you had switched to a TFT panel with a LED backlight 10 years ago would probably have made up for the manufacturing emissions several times over... You could've bought used, too.

Like, yeah, I can absolutely sympathize with repairing things, I do it myself a lot, but come on. Don't pretend this is anything other than obstinacy at this point.

[real.finder]

Groucho, you are absolutely correct. Keeping my old hardware I do hardly sacrifice anything, and I keep my carbon dioxide footprint low. Sustainability, repair instead of throwing things into the trash bin...


Correct me if I am wrong, but isn't the old hardware (especially old CPU and GPU) need more power to do the same operation that can be done in new hardware? which mean more energy and heat which mean again more carbon dioxide footprint?

[Groucho2004]

Correct me if I am wrong, but isn't the old hardware (especially old CPU and GPU) need more power to do the same operation that can be done in new hardware? which mean more energy and heat which mean again more carbon dioxide footprint?Yes, especially a CRT consumes a lot more power than a modern LCD/LED screen as TheFluff already mentioned.

[MeteorRain]

Yes, Core and Nehalem burns more coal to run. Nehalem idles at 100w while i7-9700k idles at 40w. When they are loaded they use more electric.

[Groucho2004]

Yes, Core and Nehalem burns more coal to run. Nehalem idles at 100w while i7-9700k idles at 40w. When they are loaded they use more electric.manolito uses a P3 Coppermine from around 2000 (130 nm manufacturing process).

[real.finder]

Yes, especially a CRT consumes a lot more power than a modern LCD/LED screen as TheFluff already mentioned.

yes but not like old cpu vs new one, old CRT will need around 100 W but LCD need maybe around 50 W, but if you use some old cpu (non-SSE2) to did some filtering it will use around 100 W for many Hours for whole video, but same filter and video in Modern cpu (even sandy bridge) will be only few minutes with that 100 W

[Groucho2004]

yes but not like old cpu vs new one, old CRT will need around 100 W but LCD need maybe around 50 WIt depends, I guess. My old 32 inch Sony CRT used about 130 W, my current Panasonic 42 inch LCD (which is already 7 years old) 35 W.

[Groucho2004]

Coppermine is a 180 nm processor (wiki (https://en.wikipedia.org/wiki/List_of_Intel_Pentium_III_microprocessors#%22Coppermine%22_(180_nm))).Ooops, got that mixed up with Tualatin methinks.

[Atak_Snajpera]

on a modern 8x 5 GHz 95 W processor
Haha! More like 300+ W.

[StainlessS]

To Whom it may concern,

# LSMASHSource_x86_20200118(HolyWu).dll

LSMASHVideoSource("...") # Access Violation, it dont like the ellpisis, [also bad with 1 or 2 dots]

[LigH]

Who ever would test it without a proper filename :D ... in the DVD Studio I worked for, long ago, we called this class of tests "janitor tests", because a janitor would have no clue which kind of parameter is sensible. Today it would be called "fuzzing", I believe :p

[StainlessS]

Thanx HolyWu.
Of course '...' filename should never be used, but Access Violations should also never occur.
[I just checked AviSource("...") just to see if same happened, nope, no A.V. all OK].

Ah LigH, reminiscing about his days as a janitor, such a dreamer :)
[Fuzzing, also User Friendly, or Idiot Proofing(my favourite)]

EDIT: Fuzzing, automated providing of a mixture of random inputs intention to catch problems, [did not know that, not quite Idiot Proofing]:- https://en.wikipedia.org/wiki/Fuzzing

[hello_hello]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200207/L-SMASH-Works_20200207.7z
LWLibav: Restore the functionality of opening index file as source.

Awesome! Thank you!

[dREV]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200207/L-SMASH-Works_20200207.7z


Update to FFmpeg-20200207-343ccfc.
LWLibav: Restore the functionality of opening index file as source.


:eek: It's working on MeGUI's One-Click! Testing it out and so far the index file size is waaaay smaller too.

Thanks for including it back. :D

[FranceBB]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200207/L-SMASH-Works_20200207.7z


Update to FFmpeg-20200207-343ccfc.
LWLibav: Now only certain frames are marked as interlaced internally when enable repeat control. Although it had no effect on the decoded pictures whether the stream contain repeat flags or not, it did affect the value of frame property _FieldBased in VapourSynth for streams containing repeat flags.
LWLibav: Improve the logic of frame rate deduction.
LWLibav: Restore the functionality of opening index file as source.
AviSynth/LibavSMASH: Fix crash when open an invalid file path.


Thank you for the new version! :)

[Taurus]

@HolyWu
As always: :thanks:
A big Thank You from me.

[Nico8583]

Hi, the same DLL is used for both AviSynth and VapourSynth ? Thank you.

[poisondeathray]

Hi, the same DLL is used for both AviSynth and VapourSynth ? Thank you.

Yes , same dll

[Nico8583]

Yes , same dll
Thank you, I'll try it !

[gpower2]

Hey guys!

I run into a problem with LSmashSource by using the LWLibavAudioSource with AviSynth+ 3.5

The resulting audio script has significantly smaller length from the original file. (03:17 instead of 04:07)
The original audio track properties are:

Audio
ID : 1
Format : WMA
Format version : Version 2
Codec ID : 161
Codec ID/Info : Windows Media Audio
Description of the codec : Windows Media Audio 9.2 - 20 kbps, 32 kHz, mono (A/V) 1-pass CBR
Duration : 4 min 7 s
Bit rate mode : Constant
Bit rate : 1 411 kb/s
Channel(s) : 1 channel
Sampling rate : 32.0 kHz
Bit depth : 16 bits
Stream size : 41.6 MiB (21%)
Language : English (Great Britain)

After testing different versions, I finally managed to pinpoint the version that broke this particular script:

commit 2f9d49914f42c2ffe62a9fc20b48de4d3657ffd9 [revision 921]
Author: Yusuke Nakamura <muken.the.vfrmaniac@gmail.com>
Date: Tue Dec 20 07:21:13 2016 +0900

lwindex: Consider draining mode for actual audio decoding.


All the versions from r921 till the latest 20200207 have the wrong result.

I also tested with FFMS2 and I had no problem with that.

Could someone take a look?

Thank you very much! :)

[gpower2]

Hi!

Unfortunately I can't share the file here, and you have disabled private messages here.

Is there any other channel I can reach you in order to share the file with you?

[gpower2]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200322/L-SMASH-Works_20200322.7z


Update to FFmpeg-20200322-ea46b45.
Update to dav1d 0.6.0.
AviSynth: Add Meson build system and fix compilation for non-Windows.
lwindex: Fix incorrect frame length being written into index file when decoding WMAv2.


I confirm that the issue I had with WMAv2 is now fixed!

Thank you so much for the swift resolve HolyWu! :)

[Morku]

Can someone help me, please?
When I load this kind of video (e.g. in VirtualDub2) with l-smash as avs: https://www.dropbox.com/s/nq65dc8ha0ogww7/x.mov?dl=1

using:
vid = LSMASHVideoSource("x.mov", format="YUV420P10")
aud = LSMASHAudioSource("x.mov")

AudioDub(vid, aud)


the audio is distorted. I am using latest HolyWu build. What I am doing wrong?
Thank you!

[LigH]

Video: AVDH 1280x720 29.97fps 97223kbps [V: Apple Video Media Handler [eng] (dnxhd dnxhr hqx, yuv422p10le, 1280x720, 97223 kb/s)]
Audio: PCM 48000Hz stereo 1536kbps [A: Apple Sound Media Handler [eng] (pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s)]

Codec ID: sowt

My guess would be that the endianness of the PCM audio was not correctly respected.

[qyot27]

AviSynth: Add Meson build system and fix compilation for non-Windows.
A couple of things I found when trying to actually build/use LSMASHSource on Ubuntu (although I'm pretty sure I also saw it do these on macOS and FreeBSD; I just haven't tested whether the below fixes it on those systems):

Without an install prefix set, it installs to what I guess is meson's default $libdir, so it gets installed to /usr/local/lib/x86_64-linux-gnu/avisynth/ instead of /usr/local/lib/avisynth. Since AviSynth+ installs a pkg-config file with the relevant information, it can pretty much mirror the meson.build for VapourSynth that uses pkg-config to set those values. Or does that cause some kind of conflict with MSVC builds?

The plugin entry point isn't visible, so AviSynth+ can't load it. Setting -fvisibility=default in CPPFLAGS fixed it, although since meson.build has the gnu_symbol_visibility field, that might be a better place for it, or the exact symbols could be marked visible in the source.

[sl1pkn07]

in ubuntu pkg-config see the path /usr/local/lib/pkgconfig by default?

if is no, you need set PKG_CONFIG_LIBDIR or PKG_CONFIG_PATH pounts to "/usr/local/lib/pkgconfig:" for meson can look at it

EDIT: good cat about -fvisibility=default !

edit2: patch


diff --git a/AviSynth/meson.build b/AviSynth/meson.build
index b20c0d4..97c0255 100644
--- a/AviSynth/meson.build
+++ b/AviSynth/meson.build
@@ -57,6 +57,7 @@ sources = [
includes = include_directories('../include')

deps = [
+ dependency('avisynth'),
dependency('liblsmash'),
dependency('libavcodec', version : '>=58.18.0'),
dependency('libavformat', version : '>=58.12.0'),
@@ -78,5 +79,5 @@ shared_module('lsmashsource', sources,
include_directories : includes,
install : true,
install_dir : join_paths(get_option('libdir'), 'avisynth'),
- gnu_symbol_visibility : 'hidden'
+ gnu_symbol_visibility : 'default'
)

[qyot27]

I'm logged in under Windows at the moment, but as I do have WSL with Ubuntu 19.10 set up,
$ pkg-config --variable pc_path pkg-config
/usr/local/lib/x86_64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig
So yes, it sees both (and more), with the path meson used at the front. But it appears like the reason it picks that one first for installing liblsmashsource.so is because while FFmpeg and L-SMASH are both detected via pkgconfig, AviSynth is not. As soon as I hook AviSynth up into the pkg-config queries in meson.build¹, it does install to /usr/local/lib/avisynth just like ConvertStacked does.

¹diff --git a/AviSynth/meson.build b/AviSynth/meson.build
index b20c0d4..75a92fa 100644
--- a/AviSynth/meson.build
+++ b/AviSynth/meson.build
@@ -56,7 +56,10 @@ sources = [

includes = include_directories('../include')

+avisynth_dep = dependency('avisynth').partial_dependency(compile_args : true, includes : true)
+
deps = [
+ dependency('avisynth'),
dependency('liblsmash'),
dependency('libavcodec', version : '>=58.18.0'),
dependency('libavformat', version : '>=58.12.0'),
@@ -77,6 +80,6 @@ shared_module('lsmashsource', sources,
dependencies : deps,
include_directories : includes,
install : true,
- install_dir : join_paths(get_option('libdir'), 'avisynth'),
- gnu_symbol_visibility : 'hidden'
+ install_dir : join_paths(avisynth_dep.get_pkgconfig_variable('libdir'), 'avisynth'),
+ gnu_symbol_visibility : 'default'
)

Make sure to check the .so with nm -D to make sure that AvisynthPluginInit3 is exported.

[sl1pkn07]

└───╼ nm -D /usr/lib/avisynth/liblsmashsource.so | grep Avis
000000000001de70 T AvisynthPluginInit3



pkgconfig is not used to know where to install the library, it is used to find where the libraries installed on the system for build the projects

you need set manually where install the libraries in meson with `--libdir` (or `--libexecdir`) and `--prefix` if you want install in a custom path

see https://mesonbuild.com/Configuring-a-build-directory.html

[qyot27]

I am well aware of how pkg-config and custom prefixes and install dirs work. (https://raw.githubusercontent.com/qyot27/mpv/extra-new/DOCS/crosscompile-mingw-tedious.txt)

That doesn't change the fact that I said 'to mirror the meson.build for VapourSynth':
https://github.com/HolyWu/L-SMASH-Works/blob/master/VapourSynth/meson.build#L65

And that this means it goes into the same plugin directory VapourSynth itself creates and uses for the plugins that come with the upstream source. Something which the meson.build for AviSynth is not doing at the current time.

[sl1pkn07]

Installing liblsmashsource.so to /usr/local/lib/avisynth


without setting libdir/prefix. but my avisynth (include .pc) is installed in /usr/lib


└───╼ yay -Ql avisynthplus-git
avisynthplus-git /etc/
avisynthplus-git /etc/avisynth/
avisynthplus-git /etc/avisynth/avisynth.conf
avisynthplus-git /usr/
avisynthplus-git /usr/include/
avisynthplus-git /usr/include/avisynth/
avisynthplus-git /usr/include/avisynth/avisynth.h
avisynthplus-git /usr/include/avisynth/avisynth_c.h
avisynthplus-git /usr/include/avisynth/avs/
avisynthplus-git /usr/include/avisynth/avs/alignment.h
avisynthplus-git /usr/include/avisynth/avs/capi.h
avisynthplus-git /usr/include/avisynth/avs/config.h
avisynthplus-git /usr/include/avisynth/avs/cpuid.h
avisynthplus-git /usr/include/avisynth/avs/minmax.h
avisynthplus-git /usr/include/avisynth/avs/posix.h
avisynthplus-git /usr/include/avisynth/avs/types.h
avisynthplus-git /usr/include/avisynth/avs/win.h
avisynthplus-git /usr/lib/
avisynthplus-git /usr/lib/libavisynth.so
avisynthplus-git /usr/lib/pkgconfig/
avisynthplus-git /usr/lib/pkgconfig/avisynth.pc


my /usr/local is complety pruned, only system directories


└───╼ LC_ALL=C pacman -Qo /usr/local
/usr/local/ is owned by filesystem 2019.10-2


in avisynth.pc

└───╼ cat /usr/lib/pkgconfig/avisynth.pc |grep ^libdir
libdir=${exec_prefix}/lib


meson not know the variable '${exec_prefix}'

so yes, i think is working

[Morku]

Video: AVDH 1280x720 29.97fps 97223kbps [V: Apple Video Media Handler [eng] (dnxhd dnxhr hqx, yuv422p10le, 1280x720, 97223 kb/s)]
Audio: PCM 48000Hz stereo 1536kbps [A: Apple Sound Media Handler [eng] (pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s)]

Codec ID: sowt

My guess would be that the endianness of the PCM audio was not correctly respected.

And is there something I can change? Or just wait for a fix?

[poisondeathray]

And is there something I can change? Or just wait for a fix?

Use another audio source filter for now as a temporary workaround

eg

vid = LSmashVideoSource("x.mov")
aud = FFAudioSource("x.mov")

AudioDub(vid,aud)


Actually L-Smash works, if you use LWLibavAudioSource instead of LSmashAudioSource. It produces an index. But not sure why LSmashAudioSource fails



vid = LSmashVideoSource("x.mov")
aud = LWLibavAudioSource("x.mov")

AudioDub(vid,aud)

[Morku]

wow, I dodn't expect a solution that quick.
Yes, the new "temporary fix" build working fine :) No distorted audio. Thank you.

The FFAudioSource workaround is also doing the job, but indexing takes really a long time here.

[LigH]

Nice to know that I guessed well! :D

[kedautinh12]

New patched for ffmpeg 4.2.2
https://gist.github.com/maki-rxrz/5a7a2c789e4369fa34853b5358fb8a29

[bruno321]

@xp users try this: https://www.mediafire.com/folder/17q7t31dib18t/LSMASH-Works


Was this the last build for XP users? I'm trying it now with latest AVS+ and I'm getting "system exception -- access violation".

(If there's no LSMASHVideo for XP, what's a good alternative that would allow me to feed an m2ts directly into avspmod? DirectShowSource is giving me "no video")

[videoh]

If there's no LSMASHVideo for XP, what's a good alternative Update to Win10.

[bruno321]

Ah thanks, I hadn't thought of that.

Any other answers?

[MeteorRain]

Windows XP does not have the functionality to correctly run modern applications.

In C++ application, static variables need to be safely initialized to properly run. It needs a large thread local storage, which Windows XP doesn't have. Of course you can disable thread safe init and re-compile, which allows it to run. But then you'll suffer thread-unsafe init and it will potentially give you incorrect result silently.

For now, most XP compatible filters are provided giving the notice that you are willing to take the risk of unreliable running results.

TL;DR: Recompile it with "/Zc:threadSafeInit-" option, make it unsafe, and it should work for you.

[bruno321]

Thanks for the explanation. For the moment, for uninteresting reasons, I can't update the XP installation. What I've been doing is either mux into an mkv and use ffvideosource, or use the very old DGAVCDec, which doesn't always work. I was wondering if there was a better alternative in the XP scenario, but perhaps there isn't...?

[Groucho2004]

If there's no LSMASHVideo for XP, what's a good alternative that would allow me to feed an m2ts directly into avspmod?Here's (http://download1525.mediafire.com/ohpjpgk11avg/oi31n1lcrzdvkno/ffms2_r1315%2B119-avs%2Bvsp_lastxp.7z) a link to qyot27's last XP-compatible FFMS2 C-plugin.

Edit - The best alternative for XP would be an older version (2052?) of DG's source filter for Nvidia cards but I don't know if he would be willing to make it available. Also, it's not free but the few bucks he charges are well worth it.

[bruno321]

Thanks. I already use ffvideosource succesfully, but then I feed it a demuxed mkv. In the avisynth.nl page I see that ffvideosource also accepts m2ts files, but I read that

"M2TS, TS: Seeking seems to be off a few frames here and there"

which makes me a bit anxious.

[Groucho2004]

"M2TS, TS: Seeking seems to be off a few frames here and there"

which makes me a bit anxious.I agree. In that case there's just DGDecNV as far as I can see.

[videoh]

The best alternative for XP would be an older version (2052?) of DG's source filter for Nvidia cards but I don't know if he would be willing to make it available. Also, it's not free but the few bucks he charges are well worth it. Thanks for the mention, Groucho! I thought DGDecNV 2053 should work on XP. Does it not? Also, 2052 has always been available in the binaries area.

[videoh]

"M2TS, TS: Seeking seems to be off a few frames here and there"

which makes me a bit anxious. Shouldn't matter if you are just transcoding with linear access.

[StainlessS]

You could just do away with ffms and LSmash altogether, this does a faily good job, never (well rarely) problems,
just convert to AVI and have pretty much 100% frame accuracy and zero problems.

DEMO.BAT

REM https://forum.doom9.org/showthread.php?p=1908619#post1908619
REM We DO NOT LIKE SPACES [EDIT: nor accents] IN FILE NAMES (REM == REMark ie comment)

setlocal

REM Where to Find ffmpeg
set FFMPEG="C:\BIN\ffmpeg.exe"

REM Where to get INPUT files, No terminating Backslash, "." = current directory (ie same as dir .bat file)
set INDIR="."

REM Where to place OUTPUT files, No terminating Backslash. "." would be same as .bat file, ".\OUT" = OUT folder in same directory as bat file.
set OUTDIR="D:"

REM Below, can add INPUT extensions as eg *.WEBM (SPACE separated, Batch Processes all INPUT type files in INDIR)
FOR %%A IN (*.AVI *.MKV *.MP4 *.MOV *.QT *.3GP *.DVB *.VOB *.MPG *.MPEG *M2P *.PS *.TS *.DIVX *.XVID *.FLV *.WMV *.ASF *.MXF) DO (

REM ****** Un-REM [ie SELECT] ONLY one of below lines, Comment out ie REM the remaining lines.
%FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec huffyuv -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec magicyuv -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec v410 -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec r210 -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec copy -acodec copy "%OUTDIR%\%%~nxA.MKV"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo -acodec copy "%OUTDIR%\%%~nxA.MKV"
REM %FFMPEG% -i "%INDIR%\%%A" -vn -acodec pcm_s16le "%OUTDIR%\%%~nxA.WAV"
REM *****************************************************************************************.

)
Pause

REM ... Above Command lines, What they do if UnREM'ed (and all others REM'ed, UnREM=UnCOMMENT ) :
REM (1) Convert Video to UtVideo lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM (2) Convert Video to HuffYUV lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM (3) Convert Video to MagicYUV lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM (4) Convert Video to v410 Uncompressed 4:4:4 10-bit lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM (5) Convert Video to r210 Uncompressed RGB 10-bit lossless, Convert Audio to 16 bit PCM audio (output AVI).
REM (6) Remux, copy both video and audio (output MKV - see file extension at the end of the Remux line, ie MKV).
REM (7) Convert Video to UtVideo lossless, copy audio (output MKV).
REM (8) Skip any video, Convert Audio to 16 bit PCM (output WAV).
REM *****************************************************************************************.
REM In the UnREM'ed command Line [ie without a preceding REM]:-
REM '-vcodec utvideo' means convert video using utvideo codec.
REM '-vcodec copy' means copy video rather than convert.
REM '-vn' means no video output, use instead of eg '-vcodec utvideo'
REM '-acodec pcm_s16le' means convert audio using pcm_s16le codec.
REM '-acodec copy' means copy audio rather than convert.
REM '-an' means no audio output, use instead of eg '-acodec pcm_s16le'
REM The file extension at end of the line determines output container, eg '.AVI'
REM
REM From Command line with ffmpeg somewhere in your environment PATH:-
REM 'ffmpeg -codecs >D:\ffmpeg_codecs.txt'
REM Writes a txt file of ffmpeg available codecs to D:\
REM At start of txt file it shows a 'legend' or 'key' for codecs that can be used for video and audio,
REM use only ENCODING SUPPORTED codecs, prefereably LOSSLESS.
REM Supported codecs will vary with ffmpeg version.
REM *****************************************************************************************.

EDIT: Best make seperate BAT/CMD files for your required options.
EDIT: You may be able to remux eg WMV into eg MP4 (or whatever) does not necessarily mean that it will play OK in your player.
EDIT: At the very least you could give it a try instead of starting another thread on "How to open this difficult file" [life aint forever].
EDIT: I still use DGIndex for DVD VOB (otherwise almost always this, unless some kind of test scenario).

[FranceBB]

Was this the last build for XP users? I'm trying it now with latest AVS+ and I'm getting "system exception -- access violation".

(If there's no LSMASHVideo for XP, what's a good alternative that would allow me to feed an m2ts directly into avspmod? DirectShowSource is giving me "no video")

AFAIK the last XP compatible version of LSMASH is the 2015-03-16 build you can find here: https://www.mediafire.com/folder/17q7t31dib18t/LSMASH-Works

It works fine, however there's a catch: it only supports 8bit planar, so if you index an high bit depth file (10bit or higher), it's gonna index it and output 16bit interleaved instead, which ain't a big deal, since you can convert it back to 16bit planar anyway, but still...
Besides, since it's quite old, it might not be able to index new files like AV1 and other things. Your best shot for pretty much everything other than some particular cases is ffms2 C plugin which has been updated by qyot27 for quite some time before he/she dropped support: http://www.mediafire.com/file/oi31n1lcrzdvkno/ffms2_r1315%252B119-avs%252Bvsp_lastxp.7z/file

(I'm like 99% sure he's a dude, but he has a girl as an avatar so I'll leave the 1% margin open for doubt/discussion)

ffms2 will be able to decode more things and it also supports planar high bit depth, which means that if you index a 10bit planar source, you'll get a 10bit planar output and so on. :)

Bonus bit: CoRoNe is still compiling ffmpeg for Windows XP x86, so whenever you can't index something, there's always ffmpeg lossless (huffyuv or lagarith or utvideo or ffv1 or whatever). I also asked him to compile ffms2 c plugin with his very own changes to ffmpeg so that we can keep it XP-compatible; he didn't reply yet but I hope he will do it.

https://rwijnsma.home.xs4all.nl/files/ffmpeg/?C=M;O=D

Update to Win10.

You mean "downgrade" to Windows10 hahaha
Jokes aside, I'm pretty sure everybody has a Win10 copy for productivity stuff just like I do for Visual Studio, AVID Media Composer, Protools, Davinci Resolve and also Avisynth, but I'm still attached to my beloved XP for leisure (and AVS is also leisure xD)... :D


@StainlessS... good bat, it might come in handy. Out of curiosity, do you still have your copy of XP or did you ditch it when you moved to Windows 7? :(

[videoh]

You mean "downgrade" to Windows10 hahaha I used to like you. :o

[StainlessS]

still have your copy of XP or did you ditch it when you moved to Windows 7
Still have on several machines, two never on-line (well one is a laptop which I only use for playing movie whilst at my Win7 m/c),
and another one which rarely goes on-line (except in guise of Cinnamon Mint EDIT: Usually just to update it), EDIT: that one is my regular encode machine.
Also got two or three more Laptops with XP on them, I always forget bout them, never turned them on for a couple of years.
EDIT: Probably dumped about 6 to 8 XP m/c's in last 4 or 5 years, P4 or less.

[real.finder]

(I'm like 99% sure he's a dude, but he has a girl as an avatar so I'll leave the 1% margin open for doubt/discussion)


indeed it's just his waifu avatar :p


You mean "downgrade" to Windows10 hahaha


I really see win10 better than win7 :D

but still, nothing better than winxp even if I don't use it now :sly:

[bruno321]

Was this the last build for XP users? I'm trying it now with latest AVS+ and I'm getting "system exception -- access violation".

(If there's no LSMASHVideo for XP, what's a good alternative that would allow me to feed an m2ts directly into avspmod? DirectShowSource is giving me "no video")

FWIW I don't have a problem using LWLibavVideoSource, which I hadn't read about until today.

[leon]

Using LSMASH 20200322 in VapourSynth, HW decoding doesn't seem to work. I'm trying to decode H.264 8bit in a simple script like this:

clip = core.lsmas.LWLibavSource(source=r"D:\video.mkv", prefer_hw=1)

I don't see any GPU usage. Am I doing something wrong?

And apparently "prefer_hw" can take just about any value, could you please make it fail in the case of invalid values and also if the selected decoder is not available?

[stax76]

@leon

Please provide avsmeter benchmark results like so:

https://forum.doom9.org/showpost.php?p=1912563&postcount=3407

[real.finder]

maybe make an optional cmd tools to do index for LSMASH will be a good idea, so user can now when the indexing finish, like ffindex

[leon]

My script:

LoadPlugin("C:\Program Files\MeGUI\tools\lsmash\LSMASHSource.dll")

LSMASHVideoSource("D:\video.mp4", prefer_hw=1)

GPU info:

Video card: NVIDIA GeForce GT 440
GPU version: GF108
Video memory size: 2048
OpenCL version: OpenCL 1.1 CUDA
Graphics driver version: 22.21.13.8569 (NVIDIA 385.69) / Win7 64

Result:

Frames processed: 1453 (0 - 1452)
FPS (min | max | average): 583.0 | 2460 | 1602
Memory usage (phys | virt): 62 | 63 MiB
Thread count: 18
CPU usage (average): 73%

GPU usage (average): 0%
VPU usage (average): 0%
GPU memory usage: 273 MiB

Time (elapsed): 00:00:00.907


AVS version: AviSynth+ 0.1 (r2728, MT, i386)

[stax76]

maybe make an optional cmd tools to do index for LSMASH will be a good idea, so user can now when the indexing finish, like ffindex

ffmpeg.exe -i input.avs -hide_banner


@leon

Please also post the avsmeter benchmark result for software decoding:

LSMASHVideoSource("D:\video.mp4")

AVS version: AviSynth+ 0.1 (r2728, MT, i386)

I suggest to use avisynth 3.6.

[real.finder]

ffmpeg.exe -i input.avs -hide_banner



that don't show the Progress in any way

[Groucho2004]

I have a hunch that gpu decoding with lsmash needs at least OpenCL 1.2.

Edit: What I meant to say is that the cuvid support for your card is probably insufficient gpu decoding.

[stax76]

that don't show the Progress in any way

Sure it does.

1. It's exactly what staxrip does.

2. I verified it in the console.

Try a large enough file and delete an existing index file before.

[real.finder]

Sure it does.

1. It's exactly what staxrip does.

2. I verified it in the console.

Try a large enough file and delete an existing index file before.

ok, that seems right

https://i.postimg.cc/d3HqprF8/Untitled.png (https://postimages.org/)

edit: thanks

edit2: but I think small cmd tools that can do this will be better :)

[leon]

Please also post the avsmeter benchmark result for software decoding

I did software decoding within VS and the result was the same for both, besides it says that the GPU utilisation is 0 anyway.

I have a hunch that gpu decoding with lsmash needs at least OpenCL 1.2.

My guess also was that my GPU is just too old for this. Maybe I'll try it on my iGPU (QSV).
And if that's the case maybe notify the user?

[Groucho2004]

My guess also was that my GPU is just too old for this.See my edit above. I meant to say cuvid, not OpenCL.

[stax76]

but I think small cmd tools that can do this will be better

Somebody has to build and maintain this for avisynth and for vapoursynth and for x86 and for x64, a lot of work...

Piping tools might support this or accept feature requests.

[leon]

I did test it with other videos and video formats. Also I get HW decoding with LAV Video Decoder (CUVID) for the same video.

General
Complete name : D:\video.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/avc1)
File size : 13.0 MiB
Duration : 48 s 482 ms
Overall bit rate mode : Variable
Overall bit rate : 2 250 kb/s
Encoded date : UTC 2011-03-02 16:59:19
Tagged date : UTC 2011-03-02 16:59:19

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 48 s 482 ms
Bit rate : 2 000 kb/s
Maximum bit rate : 31.3 Mb/s
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Original frame rate : 29.970 (29970/1000) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.072
Stream size : 11.7 MiB (90%)
Writing library : x264 core 95
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x101 / me=hex / subme=1 / psy=1 / psy_rd=0.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=18 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=2000 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Encoded date : UTC 2011-03-02 16:59:19
Tagged date : UTC 2011-03-02 16:59:19
Codec configuration box : avcC

Audio
ID : 1
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 48 s 390 ms
Bit rate mode : Variable
Bit rate : 125 kb/s
Maximum bit rate : 199 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 44.1 kHz
Frame rate : 43.066 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 740 KiB (6%)
Encoded date : UTC 2011-03-02 16:59:19
Tagged date : UTC 2011-03-02 16:59:19

Other #1
ID : 65536
Type : Hint
Format : RTP
Codec ID : rtp
Duration : 48 s 343 ms
Bit rate mode : Variable
Encoded date : UTC 2011-03-02 16:59:19
Tagged date : UTC 2011-03-02 16:59:19

Other #2
ID : 65537
Type : Hint
Format : RTP
Codec ID : rtp
Duration : 48 s 482 ms
Encoded date : UTC 2011-03-02 16:59:19
Tagged date : UTC 2011-03-02 16:59:19



AVSMeter 3.0.0.4 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 0.1 (r2728, MT, i386) (0.1.0.0)
[h264 @ 003DEFC0] Reinit context to 1280x720, pix_fmt: yuv420p
[h264_cuvid @ 03F85040] Cannot load cuvidGetDecodeStatus
[h264_cuvid @ 03F85040] Failed loading nvcuvid.
Creating lwi index file 100%
[AVIOContext @ 003B8680] Statistics: 13242701 bytes read, 28 seeks
[h264 @ 03E2B8C0] Reinit context to 1280x720, pix_fmt: yuv420p
[h264_cuvid @ 03E2D040] Cannot load cuvidGetDecodeStatus
[h264_cuvid @ 03E2D040] Failed loading nvcuvid.
[h264 @ 03E2D400] Reinit context to 1280x720, pix_fmt: yuv420p

Number of frames: 1453
Length (hh:mm:ss.ms): 00:00:48.482
Frame width: 1280
Frame height: 720
Framerate: 29.970 (30000/1001)
Colorspace: i420
[AVIOContext @ 03EF6E00] Statistics: 98362 bytes read, 2 seeks


@Groucho2004

I don't really know whether it's related, but what's your Nvidia driver version?

[Groucho2004]

@groucho2004

i don't really know whether it's related, but what's your nvidia driver version?436.48

[real.finder]

Somebody has to build and maintain this for avisynth and for vapoursynth and for x86 and for x64, a lot of work...

Piping tools might support this or accept feature requests.

or better it work with the video directly and only need LSMASHSource.dll so then no need to update it every time

[leon]

Well, that version is not available for my card. Thanks anyway.

[kedautinh12]

L-Smash Source meet error
https://forum.doom9.org/showthread.php?p=1914021#post1914021

[Atak_Snajpera]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200531/L-SMASH-Works_20200531.7z


Update to FFmpeg-20200530-4391289.
Update to dav1d 0.7.0.
AviSynth: Properly handle libdir in meson and plugin's entry point for non-Windows.
VideoSource: Try to fix more dropped frame cases in H.264 stream.


Thank you! Wonky_monkey's sample is now correctly decoded in non-linear mode
https://i.postimg.cc/K8bMHmKd/Untitled-1.png

[kedautinh12]

Thanks

[edison]

Does L-SMASH Source support 8K HEVC hw decoding on NVIDIA Turing GPU now?

Ok, I had the answer: yes.

but its speed is very slow( 3.x fps in AVSPMod, nvidia rtx 2070, the GPU video decoder usage is about 5%).

[LigH]

This may require a MOV container with a video codec actually supporting an alpha channel. I doubt any common video codec supported in MP4 does (maybe x265 if encoded not in YUV but in GBRA based profiles, if they exist).

Please check if Videezy (https://www.videezy.com/free-video/transparent-footage) really provides "Transparent footage Stock Video Footage" as the page title claims.

PS: "walking couple" and "girl biking" are stored in ProRes 4444 with alpha channel.

Someone with Blender experience could probably produce transparent videos.

[poisondeathray]

Could someone prodive a short video sample with alpha channel in MP4 or MOV? I'm adding support for alpha channel formats and hence needing a valid sample for testing.

Thanks for adding alpha support

PNG in mov (12.1MB)

http://www.mediafire.com/file/h4e0fcprxzwaed1/spinning_dollar_sign_PNG.mov/file

[LigH]

Success (https://www.mediafire.com/file/ca2v2wsh880tv4w/crowd_run_walking_couple.mp4/file)
crowd_run = RawSource("crowd_run_2160p50.y4m").ConvertToRGB32()
walking_couple = LSMASHVideoSource("walking_Couple.mov").ConvertToRGB32().AssumeFPS(crowd_run.FrameRate)
crowd_run.Trim(0, walking_couple.FrameCount)
layer(crowd_run.Spline16Resize(1280,720), walking_couple.Spline16Resize(1280,720)).ConvertToYV12()

Using AviSynth+ r2772 and RawSourcePlus v0.01; script ran out of memory in 32-bit VirtualDub2, but 64-bit worked.

[StainlessS]

trim past end of clip [ie last frame + 1, assuming should both be same length for Layer, probably dont cause error but could have some consequence later in script]

crowd_run.Trim(0, walking_couple.FrameCount)


maybe either

crowd_run.Trim(0, walking_couple.FrameCount - 1)


or

crowd_run.Trim(0, - walking_couple.FrameCount)


EDIT:
Ignore me, below line in BLUE does nothing [ maybe should also be crowd_run=... ]

crowd_run = RawSource("crowd_run_2160p50.y4m").ConvertToRGB32()
walking_couple = LSMASHVideoSource("walking_Couple.mov").ConvertToRGB32().AssumeFPS(crowd_run.FrameRate)
crowd_run.Trim(0, walking_couple.FrameCount) # Assign to last and then ignore as assigns below Layer line to last
layer(crowd_run.Spline16Resize(1280,720), walking_couple.Spline16Resize(1280,720)).ConvertToYV12()

[LigH]

@StainlessS: OK, I did a mistake, and it didn't even matter... :o

@HolyWu: I didn't mean it's your fault that I ran out of RAM. May be caused by processing 4K UHD videos as source and the x264 encoder requesting several frames at once before starting to encode.

[outhud]

On building for VapourSynth, I get:

../video_output.c: In function ‘determine_colorspace_conversion’:
../video_output.c:613:15: error: ‘AV_PIX_FMT_Y210LE’ undeclared (first use in this
function); did you mean ‘AV_PIX_FMT_P010LE’?
613 | { AV_PIX_FMT_Y210LE, pfYUV422P10, 1 },
| ^~~~~~~~~~~~~~~~~
| AV_PIX_FMT_P010LE


Any hint to what I'm missing?

I didn't see an L-SMASH thread for Vapoursynth. Sorry if this is in the wrong place.

[DJATOM]

Most likely your build environment lacks libavutil includes or so (libavutil/pixfmt.h)

[outhud]

Most likely your build environment lacks libavutil includes or so (libavutil/pixfmt.h)

Thanks, just needed to update my libavutil as AV_PIX_FMT_Y210LE was added in February. Seems the default Ubuntu 20.04 focal repository doesn't have it.

[amayra]

there any reason for not using GitHub Actions or appveyor ?

[Boardlord]

Long time, no post :)

I'd like to ask for some help. I've just upgraded to an AMD Ryzen 4000-based laptop with integrated Vega graphics, on dGPU. On my previous laptop I had a crappy Nvidia dGPU, but it still was a tremendous help with transcoding stuff through DGIndex. With this laptop this is out of the picture obviously.

What would be the syntax with L-SMASH in Avisynth to utilize the AMD VCN core for HW decoding, to free up the CPU for x264 encoding? If it's not possible with the current compiled binaries, can I build one that includes the capability to use the VCN core? Thanks!

[Atak_Snajpera]

Long time, no post :)

I'd like to ask for some help. I've just upgraded to an AMD Ryzen 4000-based laptop with integrated Vega graphics, on dGPU. On my previous laptop I had a crappy Nvidia dGPU, but it still was a tremendous help with transcoding stuff through DGIndex. With this laptop this is out of the picture obviously.

What would be the syntax with L-SMASH in Avisynth to utilize the AMD VCN core for HW decoding, to free up the CPU for x264 encoding? If it's not possible with the current compiled binaries, can I build one that includes the capability to use the VCN core? Thanks!

HW Decoding is not available on AMD GPU. Blame AMD for being stubborn and not providing api to their decoder like Intel or NVidia.

[Boardlord]

HW Decoding is not available on AMD GPU. Blame AMD for being stubborn and not providing api to their decoder like Intel or NVidia.Hmm, but I've seen that LAVFilters can utilize the AMD GPU for HW-accelerated playback...?

Edit: through DXVA and/or D3D11 I believe, unless I've totally misunderstood.

Sent from my SM-G973F using Tapatalk

[Atak_Snajpera]

We need proper api like cuvid or quicksync not some hacks via dxva. AMD should stop acting like some special princes and allow third party programmers access to their decoder.

[jones1913]

We need proper api like cuvid or quicksync not some hacks via dxva. AMD should stop acting like some special princes and allow third party programmers access to their decoder.

The API exists: AdvancedMediaFramework (AMF) (https://gpuopen.com/advanced-media-framework)
However, AMF decoder is not yet integrated in ffmpeg (but encoder is): https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/199

Btw. on Linux encoding and decoding works with ffmpeg through vaapi (https://trac.ffmpeg.org/wiki/Hardware/VAAPI).

[real.finder]

there are frame properties PR by Asd-g since a week https://github.com/HolyWu/L-SMASH-Works/pull/12

I think HolyWu is not active these days, so anyone else can build it?

[StvG]

Here (https://cloud.owncube.com/s/XBkpwmxAN2EByaz) - dav1d 0.7.1, ffmpeg@9b72cea446, zlib 1.2.11, xxHash v0.8.0.

[FranceBB]

Thank you!! :D

[kedautinh12]

Thanks

[real.finder]

Here (https://cloud.owncube.com/s/XBkpwmxAN2EByaz) - dav1d 0.7.1, ffmpeg@9b72cea446, zlib 1.2.11, xxHash v0.8.0.

that was fast! thanks!

[kedautinh12]

Here (https://cloud.owncube.com/s/XBkpwmxAN2EByaz) - dav1d 0.7.1, ffmpeg@9b72cea446, zlib 1.2.11, xxHash v0.8.0.

I meet error when index with LSMASH, normally with FFMPEG

error: https://drive.google.com/file/d/1ZNB7IG1r0MVKlEKh829ZZNiSz5xrvZ-V/view?usp=sharing
file test: https://drive.google.com/file/d/1yHALLsohChPUegEl6Y4HYYZTcxUtLvKC/view?usp=sharing

[real.finder]

I meet error when index with LSMASH, normally with FFMPEG

error: https://drive.google.com/file/d/1ZNB7IG1r0MVKlEKh829ZZNiSz5xrvZ-V/view?usp=sharing
file test: https://drive.google.com/file/d/1yHALLsohChPUegEl6Y4HYYZTcxUtLvKC/view?usp=sharing

I think for mkv nothing better than ffms2, LSMASH is not replacement for ffms2 so it's focuses on what ffms2 fails to do

anyway, let's see what the Developers said about it

[manolito]

I don't think that it has anything to do with the MKV container... :rolleyes:
The latest HolyWu build from July 2020 does not have this issue, so this is a regression which should be fixed.

[Atak_Snajpera]

I think for mkv nothing better than ffms2, LSMASH is not replacement for ffms2 so it's focuses on what ffms2 fails to do

anyway, let's see what the Developers said about it

Yes It is. L-SMASH is superior in every way. No idea why people are still glued to non frame accurate filter. You are just asking for troubles when editing video in avisynth.

FFMS
https://i.postimg.cc/xT2VQLqF/Untitled-2.png

L-SMASH
https://i.postimg.cc/VsMP2KsC/Untitled-1.png

[StvG]

I meet error when index with LSMASH, normally with FFMPEG

error: https://drive.google.com/file/d/1ZNB7IG1r0MVKlEKh829ZZNiSz5xrvZ-V/view?usp=sharing
file test: https://drive.google.com/file/d/1yHALLsohChPUegEl6Y4HYYZTcxUtLvKC/view?usp=sharing
Try with lwlibavvideosource(..., threads=2).

I don't think that it has anything to do with the MKV container... :rolleyes:
The latest HolyWu build from July 2020 does not have this issue, so this is a regression which should be fixed.
My build with threads=2 doesn't have this issue. Obviously I'm using different building environment than HolyWu.
#The latest official build (20200728)
lwlibavvideosource("Letter - SHE'S.mkv")


Number of frames: 6518
Length (hh:mm:ss.ms): 00:04:31.855
Frame width: 1920
Frame height: 1080
Framerate: 23.976 (24000/1001)
Colorspace: i420

Frames processed: 6518 (0 - 6517)
FPS (min | max | average): 122.0 | 2021 | 448.1
Process memory usage (max): 105 MiB
Thread count: 37
CPU usage (average): 13.3%

Time (elapsed): 00:00:14.547
#my build
lwlibavvideosource("Letter - SHE'S.mkv", threads=2)


Number of frames: 6518
Length (hh:mm:ss.ms): 00:04:31.855
Frame width: 1920
Frame height: 1080
Framerate: 23.976 (24000/1001)
Colorspace: i420

Frames processed: 6518 (0 - 6517)
FPS (min | max | average): 104.0 | 2487 | 424.9
Process memory usage (max): 104 MiB
Thread count: 24
CPU usage (average): 10.8%

Time (elapsed): 00:00:15.340
#ffms2 - no seek issue
ffvideosource("Letter - SHE'S.mkv", threads=8)


Number of frames: 6518
Length (hh:mm:ss.ms): 00:04:31.855
Frame width: 1920
Frame height: 1080
Framerate: 23.976 (24000/1001)
Colorspace: i420

Frames processed: 6518 (0 - 6517)
FPS (min | max | average): 167.8 | 2205 | 1077
Process memory usage (max): 133 MiB
Thread count: 30
CPU usage (average): 29.3%

Time (elapsed): 00:00:06.049

[manolito]

Thanks for looking into it...

I can confirm that the problem disappears when "threads=2" is specified. Still I think that this should be fixed because the HolyWu builds and also FFVideoSource do not have a problem with the default value for Threads (which means to use the number of threads reported by Windows with a ceiling of 16).

[StvG]

What ffms2 version do you use? With this one (https://forum.doom9.org/showthread.php?p=1925683#post1925683) and default threads I have 6 seek issues.

[kedautinh12]

What ffms2 version do you use? With this one (https://forum.doom9.org/showthread.php?p=1925683#post1925683) and default threads I have 6 seek issues.

I thinks thread=8 very strong for weak computer, can you fixed with default thread for both ffms2 and lsmash??

[manolito]

What ffms2 version do you use? With this one (https://forum.doom9.org/showthread.php?p=1925683#post1925683) and default threads I have 6 seek issues.

Yes, I used the version you linked to from October 2020.

I got curious and made a couple of tests using different source filters with Atak's SeekTester 1.2. I used the default frame limit of 1000 frames. As the source I took the "Letter - SHE'S.mkv" (VP9 Video 1080p). My CPU is a Core i5 third generation (2 physical cores plus hyperthreading). I used the default thread values for all source filters which would be 4 for this CPU.

FFMS2 StvG from Oct. 2020:
There were 12 corrupted frames out of 1000 frames.

FFMS2 StvG from Aug. 2020:
Same

FFMS2 StvG from April 2020:
Same

FFMS2 StvG from Nov. 2019:
Same

FFMS2 C-Plugin latest 2.31 by qyot27
Perfect

FFMS2 C-Plugin by Reino:
Both available versions fail miserably. Mainly corrupted or inserted frames

FFMS2 official version 2.40 by Myrsloik:
Fails miserably. Mostly corrupted or inserted frames

DirectShowSource with LAV Filters:
Many inserted frames

DSS2Mod with LAV Filters (preroll=15):
Perfect

LWLibavVideoSource by HolyWu:
Perfect


Ok, these results are only for this specific VP9 source file. Other source formats may behave differently, but I find this very interesting. Any comments?

[kedautinh12]

DSS2Mod with LAV Filters (preroll=15):
Perfect


Can you share link download??

[poisondeathray]

Can you share link download??

http://avisynth.nl/index.php/DSS2mod

The seeking might be ok, but beware DSS2mod usually drops the last 1 or 2 frames. 1 frame in this example. You might not "see" it if the test limit was 1000 frames

[StvG]

I thinks thread=8 very strong for weak computer, can you fixed with default thread for both ffms2 and lsmash??
I should clarify that threads=8 for ffms2 from my previous post is cpu_threads/2. So you can try with that cpu_threads/2.

Yes, I used the version you linked to from October 2020.

I got curious and made a couple of tests using different source filters with Atak's SeekTester 1.2. I used the default frame limit of 1000 frames. As the source I took the "Letter - SHE'S.mkv" (VP9 Video 1080p). My CPU is a Core i5 third generation (2 physical cores plus hyperthreading). I used the default thread values for all source filters which would be 4 for this CPU.

FFMS2 StvG from Oct. 2020:
There were 12 corrupted frames out of 1000 frames.

FFMS2 StvG from Aug. 2020:
Same

FFMS2 StvG from April 2020:
Same

FFMS2 StvG from Nov. 2019:
Same

FFMS2 C-Plugin latest 2.31 by qyot27
Perfect

FFMS2 C-Plugin by Reino:
Both available versions fail miserably. Mainly corrupted or inserted frames

FFMS2 official version 2.40 by Myrsloik:
Fails miserably. Mostly corrupted or inserted frames

DirectShowSource with LAV Filters:
Many inserted frames

DSS2Mod with LAV Filters (preroll=15):
Perfect

LWLibavVideoSource by HolyWu:
Perfect


Ok, these results are only for this specific VP9 source file. Other source formats may behave differently, but I find this very interesting. Any comments?
Thanks for sharing.

[kedautinh12]

I meet many corrupted frames in STVG ver with default threads, only work with threads >= 2. But with HolyWu ver, i don't meet error with default threads:

My video test:
https://drive.google.com/file/d/17ych7Zxjh104XoVsvW-cpMEBSyAhgDj3/view?usp=sharing

My scripts:
nnedi3_rpow2(rfactor=2, cshift="spline64resizemt", fwidth=1920, fheight=1080)
F3KDB_3(range=12, y=60, cb=40, cr=40, grainy=0, grainc=0)
f3kgrain()

[manolito]

Yes, I also had to return to the HolyWu build for exactly this reason...

I believe that StvG should implement a similar logic for the number of threads which he used in his last ffms2 build:
- changed the default threads number:
* VP9 - min(CPU logical processors, 8);
* AV1 - 1 when CPU logical processors = 1, otherwise 2;
* rest - CPU logical processors;

[real.finder]

maybe it's time for new update from StvG, since there are new small update here https://github.com/Asd-g/L-SMASH-Works

[StvG]

Update (https://cloud.owncube.com/s/MXk3zBcZfNkrGF9) - included the latest commit (https://github.com/Asd-g/L-SMASH-Works/commits/master-1); ffmpeg@494f868e93; zlib 1.2.11; lsmash@18a9ed2; xxHash 0.8.0; dav1d 0.8.0; changed VP9 default threads to 2 when CPU logical processors > 2.

[kedautinh12]

Thanks

[stax76]

Update (https://cloud.owncube.com/s/MXk3zBcZfNkrGF9) - included the latest commit (https://github.com/Asd-g/L-SMASH-Works/commits/master-1); ffmpeg@494f868e93; zlib 1.2.11; lsmash@18a9ed2; xxHash 0.8.0; dav1d 0.8.0; changed VP9 default threads to 2 when CPU logical processors > 2.

vs support?

[StvG]

vs support?

Here (https://cloud.owncube.com/s/C8SBjF53JTL7s73) dualsynth ver.

[stax76]

Here (https://cloud.owncube.com/s/C8SBjF53JTL7s73) dualsynth ver.

Awesome, thanks!

[LigH]

@StvG: Are there easy-to-apply patches so that you could e.g. suggest them to media-autobuild suite?

[StvG]

It's not only patches. You need Visual C++ libraries for Avisynth but media-autobuild suite doesn't provide such. If you have them installed then using mingw could be easily used.

[LigH]

Oh, I believe I replied to the wrong thread. I was rather interested in patches for x265 like the aMod by DJATOM.

[Nico8583]

Here (https://cloud.owncube.com/s/C8SBjF53JTL7s73) dualsynth ver.
Hi, what is dualsynth version ? Does it work wit AviSynth or do I use the 12.12.2020 version ? Thank you.

[ChaosKing]

Hi, what is dualsynth version ? Does it work wit AviSynth or do I use the 12.12.2020 version ? Thank you.

The DLL works with Avisynth and Vapoursynth.

[Nico8583]

The DLL works with Avisynth and Vapoursynth.
Thank you !

[kedautinh12]

I seen new commit here, any compile update for this??
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works

[MeteorRain]

It's for my own use. You shouldn't use my version unless you know what you are doing.

[frank]

Sorry, in the latest binaries from StvG the hardware decoders don't work anymore.
With v20200728 from HolyWu they work.

[Nico8583]

Thank you HolyWu !

[real.finder]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20210421/L-SMASH-Works_20210421.7z


Update to FFmpeg 4.4.
Update to dav1d 0.8.2.
Update to nv-codec-headers 11.0.10.1.
Update mfx_dispatch to API 1.35.
Update to xxHash 0.8.0.
LWLibav: Set frame properties for AviSynth+. (Asd-g)
LWLibav: Export AviSynth variables FFSAR_NUM, FFSAR_DEN, FFSAR. (Asd-g)
lwindex: Fix crash when opening remote files. (AkarinVS)
lwindex: Hash value in FileHash field is now 64 bits long.


thanks

did you add what Asd-g forget https://github.com/HolyWu/L-SMASH-Works/pull/12#issuecomment-753911159 ?

[kedautinh12]

Thanks

[StainlessS]

Thankyou Mister Wu :) :- https://www.youtube.com/watch?v=3w3X95uWv8A

[gispos]

Thanks, this is almost the only source filter I use:)

[abyss616]

I tried to do some research on this, both in reading this thread and my own testing, but I wanted to get a more definitive answer: is the cache file created by MeGUI different than the one created when using VirtualDub/AvsPMod? Since I'm dealing with 1080i files that are several GB in size, I'd love to use the same cache file across both programs so I don't have to wait (and wait, and wait) for a new file to be generated when I go from one program to another. Thanks.

[scharfis_brain]

I typically capture my VHS-tapes three times into YUV422-10bit (Blackmagic Intensity Shuttle).
Then I load these three files in AVISynth+ using LibavSMASH and do some median() processing to get rid of recording-noise.

I really gave LibavSMASH a try. But it nearly always reindexed my files upon reloading the AVS-file into VDub. And my files are *huge* (400GB per File).
It takes about 15 minutes to reindex a single file. So reloading a AVS-file may take up to 45 minutes. This is very painful.

How can I force LibavSMASH to never re-index a file if a index is already present?

[poisondeathray]

I typically capture my VHS-tapes three times into YUV422-10bit (Blackmagic Intensity Shuttle).
Then I load these three files in AVISynth+ using LibavSMASH and do some median() processing to get rid of recording-noise.

I really gave LibavSMASH a try. But it nearly always reindexed my files upon reloading the AVS-file into VDub. And my files are *huge* (400GB per File).
It takes about 15 minutes to reindex a single file. So reloading a AVS-file may take up to 45 minutes. This is very painful.

How can I force LibavSMASH to never re-index a file if a index is already present?


It normally does not re-index by default. You might have done something to the cache file (.lwi), or location of files ?

Also, if you use MOV container for v210, you don't need an index when using LSmashVideoSource

[StainlessS]

This anything to do with it [from old docs, dont know if changed]


#LoadPlugin("LSMASHSource.dll")
##################################
## @ atrack - audio track number. Default auto. If -2, ignore audio.
## @ fpsnum, fpsden - framerate. Default auto.
## @ stacked - if true, return Stack16 format.
## (note, stacked=true requires a 16-bit color format)
## @ format - force specified output pixel format. Default auto.
## (see documentation for valid color formats)
## (if stacked = true, default "YUV420P16")
## @ cache - if true (the default), create an index file.
##
function LibavSource2(string path, int "atrack",
\ int "fpsnum", int "fpsden",
\ string "format", bool "stacked", bool "cache")
{
atrack = Default(atrack, -1)
fpsnum = Default(fpsnum, 0)
fpsden = Default(fpsden, 1)
stacked = Default(stacked, false)
cache = Default(cache, true)

format = Default(format, "")
format = (format=="" && stacked==true) ? "YUV420P16" : ""

video = LWLibavVideoSource(path,
\ fpsnum=fpsnum, fpsden=fpsden, format=format,
\ stacked=stacked, cache=cache)
return (atrack==-2) ? video: AudioDub(video,
\ LWLibavAudioSource(path, stream_index=atrack, cache=cache))
}


Maybe try with cache=false (perhaps after check [ Exist() ] on exisitng index).

EDIT: Or is that too obvious.

EDIT: Yeah, cache default true for current LWLibavAudioSource,
http://avisynth.nl/index.php/LSMASHSource/LWLibavVideoSource

[kedautinh12]

You can use LSMASHVideoSource without cache function

[kedautinh12]

Error when index .avi with L-SMASH source, original video have 25 minutes but when i index video output just 22 minutes

Example video: https://drive.google.com/file/d/1MDiKpaIc_-lCrVtPQUPAU9Gy__HykhHh/view?usp=drivesdk

[JKyle]

@HolyWu,

Plz take a look at this issue (https://github.com/HolyWu/L-SMASH-Works/issues/21) I posted the other day.
It's about wrong frame rate(playback duration) info served by LSMASHVideoSource for (some) Dolby Vision video streams.

[Katie Boundary]

I'm having an interesting problem. I have 32-bit AVIsynth 2.60 installed on both my Win7 laptop and the WinXP x64 partition on my desktop. I have an MP4 file and an AVIsynth script on an external hard drive (to eliminate any problems resulting from differences between files). The newest version of 32-bit Lsmashsource.dll has been extracted to the plugin folders on both computers. The laptop's avisynth can read the MP4 just fine, but my desktop's avisynth complains that there is no function named lsmashaudiosource, LWLibavAudioSource, ffms2source, etc.??

[kedautinh12]

Your video is VFR (Variable Frame Rate). The returned frame count from the source filter is correct. But the returned frame rate is higher than the average frame rate, hence the duration becomes shorter. However that's not an issue at all if you know how to correctly process VFR videos.

my mediainfo wasn't reported that vfr, now i used fpsnum=25, fpsden=1 and that correct duration, sr for your waste time

[LigH]

@ Katie Boundary: Use AVSMeter (https://forum.doom9.org/showthread.php?t=174797) (AVSMeter.exe -avsinfo -l) to check if AviSynth is installed correctly, in which directory it looks for plugins, which plugins are found, which functions are known (the created log file is more verbose than the screen output).

[tebasuna51]

Your video is VFR (Variable Frame Rate). The returned frame count from the source filter is correct. But the returned frame rate is higher than the average frame rate, hence the duration becomes shorter. However that's not an issue at all if you know how to correctly process VFR videos.

I can't understand that. My test show:

1) Load Kanjani8.avi in VirtualDub2:
38902 frames, 25 fps -> Duration 24:56
sequence of frames:
KDPPPPPPPPPPKDKDKDKPPPPPPPPPPPPPPPPPPPDPDPDP...
K=Key frame, D=Drop or Duplicated frame

2) Load LWLibavVideoSource("C:\tmp\Kanjani8.avi") in VirtualDub2:
34239 frames, 25 fps -> Duration 22:49
Drop/Duplicated frames deleted, but the fps is not changed then the duration is shorter.
It is not than you say or the help show:
+ fpsnum (default : 0)
Output frame rate numerator for VFR->CFR (Variable Frame Rate to Constant Frame Rate) conversion.
If frame rate is set to a valid value, the conversion is achieved by padding and/or dropping frames at the specified frame rate.
Otherwise, output frame rate is set to a computed average frame rate and the output process is performed by actual frame-by-frame

3) Load LWLibavVideoSource("C:\tmp\Kanjani8.avi", fpsnum=25, fpsden=1) in VirtualDub2:
38902 frames, 25 fps -> Duration 25:56
The Drop/Duplicated frames are preserved. Now is correct.

4) Load FFVideoSource("C:\tmp\Kanjani8.avi", timecodes = "C:\tmp\Kanjani8.txt") in VirtualDub2:
34239 frames, 22 fps -> Duration 25:56
FFVideoSource output the average fps to preserve duration.
And output the timecodes for a perfect sync with audio in a mkv VFR.
I don't know how output timecodes with LWLibav.

5) Load FFVideoSource("C:\tmp\Kanjani8.avi", fpsnum=25, fpsden=1) in VirtualDub2:
38902 frames, 25 fps -> Duration 25:56
The Drop/Duplicated frames are preserved. The same than LWLibav.

And MediaInfo can't detect VFR in avi files.
With Avinaptic (http://fsinapsi.altervista.org/) we can detect Null/Drop/Duplicated frames:
Video track

FourCC XVID/XVID
Resolution 1280 x 720
Frame aspect ratio 16:9 = 1.777778
Pixel aspect ratio 1:1 = 1
Display aspect ratio 16:9 = 1.777778
Vprp aspect ratio 71:40 = 1.775 (~16:9)
Framerate 25 fps
Total frames 38,902
Stream size 559,355,390 bytes (533.442869 MiB)
Bitrate 2875.715336 kbps
Qf 0.124814
Key frames 264 (0; 12; 14; 16; 18; ... 38763)
Null frames 4,663 (1; 13; 15; 17; 38; ... 38837)
...

[kedautinh12]

Thanks for your info but how to make timecodes file???

[tebasuna51]

To make timecodes with FFVideoSource("C:\tmp\Kanjani8.avi", timecodes = "C:\tmp\Kanjani8.txt")

Now recode only 34239 frames and after add timecodes with mkvmerge/mp4fpsmod for mkv/mp4

[kedautinh12]

Thanks

[FranceBB]

Hi there, HolyWu, I noticed that you have archived your repository, so I can't open any new issue on GitHub, therefore I might as well post it here.

I think this is something of concern that has to be addressed on developers side: https://forum.doom9.org/showthread.php?p=1943307

[GMJCZP]

I am having a problem with, at least, the latest version of Lsmash, at first I thought it was a neo_f3kdb problem. The discussion is at the following link:

Here (https://forum.doom9.org/showthread.php?p=1942911#post1942911)

Thanks.

Edit: possibly the bug was introduced from this year, I tested the 20200728 and it still works fine.

Edit2: The funny thing is that the old f3kdb works with the latest version of Lsmash, as you can see in the link.

EDIT3: problem fixed, see link.

[GMJCZP]

On the Lsmash wiki page I got an error in the script "Combining LSMASHVideoSource + LSMASHAudioSource", here is the fix:

# Combining LSMASHVideoSource + LSMASHAudioSource

#LoadPlugin("LSMASHSource.dll")
##################################
## @ atrack - audio track number. Default auto. If -2, ignore audio.
## @ fpsnum, fpsden - framerate. Default auto.
## @ format - force specified output pixel format. Default auto.
## (see documentation for valid color formats)
##
##
function LSmashSource2(string path, int "atrack",
\ int "fpsnum", int "fpsden",
\ string "format")
{
atrack = Default(atrack, 0)
fpsnum = Default(fpsnum, 0)
fpsden = Default(fpsden, 1)

format = Default(format, "")

video = LSMASHVideoSource(path,
\ fpsnum=fpsnum, fpsden=fpsden,
\ format=format)
return (atrack==-2) ? video: AudioDub(video,
\ LSMASHAudioSource(path, track=atrack))
}


It is just a parenthesis that is instead of a comma, indicated in red.

[real.finder]

Hi there, HolyWu, I noticed that you have archived your repository, so I can't open any new issue on GitHub, therefore I might as well post it here.

I think this is something of concern that has to be addressed on developers side: https://forum.doom9.org/showthread.php?p=1943307

HolyWu seems will not do any update from now on since he made the project read only https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/3#issuecomment-850781424

so maybe we need another adopter for this Project

[Atak_Snajpera]

Is hardware decoding broken in latest version?
With this script
video=LWLibavVideoSource("F:\_Video_Samples\mkv\[SS-Eclipse]_Kanon_-_Promo_(h264)_[E3CE768D] (1).mkv",cachefile="C:\Temp\RipBot264temp\job1\[SS-Eclipse]_Kanon_-_Promo_(h264)_[E3CE768D] (1).mkv.lwi",prefer_hw=3)

I'm getting
System exception - Access Violation

It used to work fine with version from year 2020

[FranceBB]

HolyWu seems will not do any update from now on since he made the project read only
so maybe we need another adopter for this Project

Ah... I see...
That's a shame...

[real.finder]

Ah... I see...
That's a shame...

there are this https://github.com/AkarinVS/L-SMASH-Works/releases

BUT, its x64 only (at least for now), and seems don't care much about AVS/AVS+ and focus only on vs

[kedautinh12]

Wow, it's support .VOB file

[JKyle]

there are this https://github.com/AkarinVS/L-SMASH-Works/releases

BUT, its x64 only (at least for now), and seems don't care much about AVS/AVS+ and focus only on vs

:thanks: for the info.

It's kind of sad that HolyWu has freezed his repo. :(
I hope he's just on a leave for whatever reason and wish that the leave won't take that long. :)

---

So I tried AkarinVS's vA.2b on StaxRip (by renaming the file name to fit into StaxRip), and it seems that it's not for AVS like real.finder pointed out.

https://i.imgur.com/ODA7JGz.png

:(

Or am I missing anything here?

According to AkarinVS's blurb on his repo,

I generally work on enhancing the VapourSynth plugin, but I will try my best to not break the AviSynth plugin.

but the filter call LSMASHVideoSource("%source_file%") definitely fails.

I suspect he probably has pruned the AVS part out of the source. :scared:

[tebasuna51]

Hi there, HolyWu, I noticed that you have archived your repository, so I can't open any new issue on GitHub, therefore I might as well post it here.

I have also a feature request. See https://forum.doom9.org/showthread.php?p=1945716#post1945716
and the equivalent feature request for FFAudioSource() and BestAudioSource()

[real.finder]

msg7086/MeteorRain said

if anyone is willing to participate in developing or maintaining any projects in this org, please let me know and I'll invite you.

before, and recently in github (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/3#issuecomment-850786923)

Let me know if I should add others to the org. My new job doesn't allow me to spend much time on those filters anymore so I'd be happy to let someone else handle the projects.

now aside from msg7086, there are me (not a programmer) and Asd-g in "Home Of AviSynthPlus Evolution" org

if anyone like to join to develop L-SMASH Source there https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works since AkarinVS seems do it only for himself, his changes can be backported with HolyWu last changes to https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works anyway

[kedautinh12]

Asd-g still don't in "Home Of AviSynthPlus Evolution" org
https://github.com/orgs/HomeOfAviSynthPlusEvolution/people

[real.finder]

Asd-g still don't in "Home Of AviSynthPlus Evolution" org
https://github.com/orgs/HomeOfAviSynthPlusEvolution/people

You cant see him there since its hidden by default unless you are a member or he made it public

[real.finder]

HolyWu last changes added https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works

[tebasuna51]

Added feature request to HomeOfAviSynthPlusEvolution/L-SMASH-Works (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/5)

We hope than this fork continue the job.

[boondoggle]

On the Lsmash wiki page I got an error in the script "Combining LSMASHVideoSource + LSMASHAudioSource", here is the fix:

# Combining LSMASHVideoSource + LSMASHAudioSource

#LoadPlugin("LSMASHSource.dll")
##################################
## @ atrack - audio track number. Default auto. If -2, ignore audio.
## @ fpsnum, fpsden - framerate. Default auto.
## @ format - force specified output pixel format. Default auto.
## (see documentation for valid color formats)
##
##
function LSmashSource2(string path, int "atrack",
\ int "fpsnum", int "fpsden",
\ string "format")
{
atrack = Default(atrack, 0)
fpsnum = Default(fpsnum, 0)
fpsden = Default(fpsden, 1)

format = Default(format, "")

video = LSMASHVideoSource(path,
\ fpsnum=fpsnum, fpsden=fpsden,
\ format=format)
return (atrack==-2) ? video: AudioDub(video,
\ LSMASHAudioSource(path, track=atrack))
}


It is just a parenthesis that is instead of a comma, indicated in red.

Uhm.. the change to comma broke the script for me!?

[StainlessS]

Uhm.. the change to comma broke the script for me!?
I think GMJCZP was saying that the comma [','] was wrong, and closing parenthesis [')'] was correct.

[kedautinh12]

Good NEWS, AkakinVS said he will update avisynth ver
https://github.com/AkarinVS/L-SMASH-Works/issues/3#issuecomment-879623789

[JKyle]

Good NEWS, AkakinVS said he will update avisynth ver
https://github.com/AkarinVS/L-SMASH-Works/issues/3#issuecomment-879623789

Wow, that's pretty good news. :cool:

[kedautinh12]

Wow, that's pretty good news. :cool:

If you had issues, you can report him

[FranceBB]

Yeah, the problem is that the FFMpeg devs just seem not to care about my bugs... -.-

https://trac.ffmpeg.org/ticket/9219 and https://trac.ffmpeg.org/ticket/9255

But the IPCM one I think can be integrated in LWLibavAudioSource.
Now, I don't know how it calls the decoding libraries, but if it's possible to include some sort of conditional to check whether it's a normal PCM or an IPCM from Sony, it would be better as it would be possible to add the relative information to the decoder so that the right library can be called.
Technically speaking, if there was a way to tell Libav what the input file is and specify a particular decoder, there would be no problems at all decoding IPCM files: https://forum.doom9.org/showthread.php?p=1942428

About the DNX one, I think we should storm from Doom9 the bug tracker of FFMpeg and insist on this issue 'cause they have to get that crap fixed.
Even nowadays I gotta use the workaround I provided but it's incredibly slow, especially for big files over the network...
It's just not feasible, it has to work out of the box...

[tebasuna51]

But the IPCM one I think can be integrated in LWLibavAudioSource.
Now, I don't know how it calls the decoding libraries, but if it's possible to include some sort of conditional to check whether it's a normal PCM or an IPCM from Sony, it would be better as it would be possible to add the relative information to the decoder so that the right library can be called.
Technically speaking, if there was a way to tell Libav what the input file is and specify a particular decoder, there would be no problems at all decoding IPCM files: https://forum.doom9.org/showthread.php?p=1942428

The check for pcm is not easy. If I well remember madshi, in eac3to, try encode to flac with many options and the most compressible is the good.

But with parameters can work fine. From NicAudio wiki (http://avisynth.nl/index.php/NicAudio):
NicLPCMSource (string, int "samplerate", int "samplebits", int "channels")
...
int samplebits = 0

Necessary for lpcm and raw files. Valid values 8/16/24/32 (also 33 (32 float) for raw) (lpcm also accept 20 and -8/-16/-24/-32. Negative values are for BluRay lpcm (big-endian)).

BTW if you need AviSynth to open your IPCM_Sample_track2.ipcm that works:

NicLPCMSource("C:\...\IPCM_Sample_track2.ipcm", 48000, -24, 2)

[FranceBB]

if you need AviSynth to open your IPCM_Sample_track2.ipcm that works:

NicLPCMSource("C:\...\IPCM_Sample_track2.ipcm", 48000, -24, 2)

Ah, so you can pass parameters to NICAudio to make it work, sweet. :)
That's good to know.

[qyot27]

Does this build of FFmpeg detect it correctly? I couldn't download the sample to verify for myself.

https://www.mediafire.com/file/7ttrj9zvnxb9z2y/ffmpeg_isomipcm_64bit.7z/file

[tebasuna51]

Does this build of FFmpeg detect it correctly?

Yes work fine:

ffmpeg.exe -i IPCM.MP4 -acodec pcm_s24le out1.wav
...
Stream #0:1(und): Audio: pcm_s24be (ipcm / 0x6D637069), 48000 Hz, stereo, s32 (24 bit), 2304 kb/s (default)

and the output is correct.

[real.finder]

Good NEWS, AkakinVS said he will update avisynth ver
https://github.com/AkarinVS/L-SMASH-Works/issues/3#issuecomment-879623789

Wow, that's pretty good news. :cool:

not for a long time https://github.com/AmusementClub/L-SMASH-Works/commit/47ebb6ed01d8107ad3b9d62d257fc2200ef9dcce

even if they say
Amusement club internal L-SMASH releases. Generally for specific needs of our members, *not* for public consumption.

that not a good sign, AkakinVS repo seems not get new updates last days unlike AmusementClub one, so I think we should better depends on https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works

[real.finder]

btw, there are avs build here https://github.com/AkarinVS/L-SMASH-Works/releases/tag/vA.3b it should work with HW decoding since last update seems not work https://forum.doom9.org/showpost.php?p=1943898&postcount=1326

as a side note, maybe this FFmpeg https://github.com/AkarinVS/FFmpeg worth pay attention

[kedautinh12]

I seen only x64 ver

[GMJCZP]

I seen only x64 ver

Exactly says:

All the released binaries are for Windows/x64.
Experimental AviSynth+ plugin is also included, but it does not include any of the VS only changes mentioned above.
Hopefully this is temporary.

[kedautinh12]

X86 here
https://github.com/AkarinVS/L-SMASH-Works/issues/3#issuecomment-882612464

Only me can't test everything about format video, audio. Need everyone test with me to development L-SMASH of AkarinVS

[GMJCZP]

X86 here
https://github.com/AkarinVS/L-SMASH-Works/issues/3#issuecomment-882612464

Only me can't test everything about format video, audio. Need everyone test with me to development L-SMASH of AkarinVS

I am getting this error message: "Script error: There is no function named 'LWLibavVideoSource'".

[qyot27]

As I commented over there, what probably happened is that it got built using meson with g++. It works, but only with a GCC build of AviSynth+.

[real.finder]

Added feature request to HomeOfAviSynthPlusEvolution/L-SMASH-Works (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/5)

We hope than this fork continue the job.

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/5#issuecomment-887420681

also everyone, Asd-g made release https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases don't know if HW decode work or not in this build

[FranceBB]

Lovely. Thanks for the heads up. ;)

[JKyle]

I've just confirmed that the new release, 20210728 (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/tag/20210728), works nicely with NVIDIA CUVID acceleration on both AviSynth+(3.7.0) and VapourSynth(R54).
:thanks: for the wonderful job!

[tebasuna51]

And the new release have the drc_scale parameter working fine for me. Thanks.

[gispos]

Can't play MPEG-TS AVC video files with it, with all previous versions there are no problems with these files.

[kedautinh12]

You need report to developer
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[JKyle]

Can't play MPEG-TS AVC video files with it, with all previous versions there are no problems with these files.

I've experienced a similar phenomenon with an MPEG-TS HEVC video file (HDR10).

For reference, here's the issue (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/6) I submitted on the GitHub repo.

[kedautinh12]

Can't play MPEG-TS AVC video files with it, with all previous versions there are no problems with these files.

Check new ver
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/tag/20210729

[gispos]

Check new ver
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/tag/20210729
Yes, it works now

[JKyle]

Ver 20210729 opens the sample HDR10 file OK but there is another issue.
Now it outputs an incorrect pixel type.

Original: YUV420P10 --(LWLibavVideoSource)--> YV12(YUV420P8) (without an explicit format setting)

I've seen the same issue with another 10-bit source (HLG HDR video) (https://4kmedia.org/travelxp-4k-hdr-hlg-sample/).

Plz have a look at my issue thread (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/6).

[FranceBB]

I cannot reproduce the problem.

I opened a YUV 4:2:2 10bit HDR PQ AppleProRes UHD 4000 nits mov source and it displays the right bit depth:

https://i.imgur.com/kzTSybM.png

I also tried with a YUV 4:2:2 10bit HDR HLG XAVC Intra Class 300 UHD 1000 nits mxf source and it displays the right bit depth again:

https://i.imgur.com/Ndpt27A.png
https://i.imgur.com/tpjMm0S.png

At this point I thought it was only happening with H.265 HEVC files, therefore I tried with a YUV 4:2:0 10bit HDR PQ H.265 HEVC HDR PQ 1000 nits .ts file and...

Video
ID : 257 (0x101)
Menu ID : 1 (0x1)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
Codec ID : 36
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Writing library : ATEME Titan File 3.7.9 (4.7.9.0)

https://i.imgur.com/S2bVVI4.png
https://i.imgur.com/oIpmuJz.png

everything was fine again.

Avisynth 3.7.1 x64
Windows 10 Enterprise x64
LSMASH version 20210729 x64

[JKyle]

CPU decoding is OK with me too.

The issue arises when hardware decoding (NVIDIA CUVID) is employed (prefer_hw=1).

You can see my tests and screenshots on the issue thread.

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/6

[FranceBB]

Ah, right. Well I can't test that as my Quadro doesn't support HEVC as it's fairly old...

[Morku]

Someone have the same? I load a .mp4 file with "Frame rate : 23.976 (24000/1001) FPS", but L-Smash Output is "Framerate : 1.5321345 (4601/3003)"

I have tested with StaxRip and MeGUI version 20210811. When I use DGIndexNV, the Framerate is fine. I tested another mp4 with same issue.
Script is just a simple "LSMASHVideoSource("file.mp4")"

[FranceBB]

Someone have the same? I load a .mp4 file with "Frame rate : 23.976 (24000/1001) FPS", but L-Smash Output is "Framerate : 1.5321345 (4601/3003)"


Nope. I can't reproduce the bug.

I made a simple:


ColorBars(848, 480, pixel_type="YV12")

ConvertFPS(23.976)

trim(0, 100)


and encoded it with the most basic settings ever in H.264, encoded the audio in AAC with NeroAAC and then I muxed everything in MP4 with MP4box:

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 2 s 169 ms
Bit rate : 11.0 kb/s
Width : 848 pixels
Height : 480 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (23976/1000) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.001
Stream size : 2.92 KiB (2%)
Writing library : x264 core 152 r2851bm ba24899
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 4 s 234 ms
Duration_LastFrame : -11 ms
Bit rate mode : Variable
Bit rate : 243 kb/s
Maximum bit rate : 256 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 126 KiB (96%)
Default : Yes
Alternate group : 1

Then I used LWLibavVideoSource to index the newly created file and... it works fine:

https://i.imgur.com/YMBf8vB.png
https://i.imgur.com/9BsKgrv.png


same goes for LSMASHVideoSource:

https://i.imgur.com/oqXvZrw.png
https://i.imgur.com/NOJz0mv.png



Would you mind uploading a sample for us?
We need to know which codec leads to your issue, not just the container...

[Morku]

Who can I provide the sample in PM?
I have found out, not all mp4 are affected, but I can reproduce on two.

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 41 min 12 s
Bit rate : 4 648 kb/s
Maximum bit rate : 22.6 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.093
Stream size : 1.34 GiB (88%)
Encoded date : UTC 2021-08-28 16:28:55
Tagged date : UTC 2021-08-28 16:28:55
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC

Audio
ID : 2
Format : E-AC-3
Format/Info : Enhanced AC-3
Commercial name : Dolby Digital Plus
Codec ID : ec-3
Duration : 41 min 12 s
Bit rate mode : Constant
Bit rate : 640 kb/s
Channel(s) : 6 channels
Channel layout : L R C LFE Ls Rs
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Stream size : 189 MiB (12%)
Language : German
Service kind : Complete Main
Encoded date : UTC 2021-08-28 16:28:55
Tagged date : UTC 2021-08-28 16:28:55

[kedautinh12]

Just give sample to L-SMASH Works developer
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[kedautinh12]

If you want private, ask @Reel_Deel (one man stand) way to contact Asd_g

[Morku]

The reason is simply I can't share copyright material public to report an issue. As I know, GitHub has no private message feature (or I can't see it).
Since the issue is just with two mp4 files (which also decode fine with FFVideoSource and DGSource), I can't create an innocent file to share.

If Reel_Deel and Asd_g would like to have a look, it would be great if they drop me a message to check. :)

[kedautinh12]

The reason is simply I can't share copyright material public to report an issue. As I know, GitHub has no private message feature (or I can't see it).
Since the issue is just with two mp4 files (which also decode fine with FFVideoSource and DGSource), I can't create an innocent file to share.

If Reel_Deel and Asd_g would like to have a look, it would be great if they drop me a message to check. :)

I mean another way not in github

[poisondeathray]

@Morku, try LWLibavVideoSource instead of LSmashVideoSource

Was file "cut" or stream copied ? Sometimes cut file can have buggy timestamps; LSmashVideoSource does not index MOV/MP4 and declares wrong framerate

[Morku]

@poisondeathray
LWLibavVideoSource helps indeed. The framerate is fine.

It was downloaded with AnyStream. No other adjustments. So I started to clickthrough StaxRip and ran into the issue. Wondering why it takes so long and files size gets unfamiliar high :)

Other files, received the same way, are fine, using LSMASHVideoSource.

EDIT: I see, I can change the Source filter to LWLibavVideoSource in StaxRip. Still wondering, because unter Settings -> Video there is a check for "Add filter to automatically correct the frame rate", would expect an AssumeFPS here, because the Logfile shows a difference in "Media Info Source File" = 23.976 and following "Source Script Info" = 1.5321345

Anyway, if it something normal, thank you for the hint.

[Balling]

"The issue arises when hardware decoding (NVIDIA CUVID) is employed"

This is normal, ffmpeg also defaults to pixel format nv12 when you decode avc with -c:v h264_cuvid -i or with -hwaccel nvdec.

You can just specify ffplay -s 1920x1080 -pix_fmt nv12 file.yuv (you also need to set the matrix at least with -vf scale=in_color_matrix) and it will work. There is no need to use yuv420p10.

Maybe it is a good idea to already move to p010 and p016. Those are now what is used on linux too.

[Katie Boundary]

I've found that LWLibavVideoSource indexes video MUCH more quickly than ffvideosource or ffmsindex does, meaning that you can use it for longer video files without programs becoming unresponsive. This, in turn, allows easier batch-indexing. I've only learned this in the past day or two and I've already replaced ffmsindex/ffvideosource with Libav in my workflow.

Lsmash just seems to be useless though.

[LigH]

LSMASHVideoSource is not at all "useless". It does not need extra indexing when it can rely on an index chunk already available in the source file. Disadvantage: The contained index must be reliable... and the source must belong to the ISO/IEC Base Media File Format (https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format) standard (e.g. MP4, MOV, 3GPP(2)).

[FranceBB]

Lsmash just seems to be useless though.

Oh... poor LSMASH... :(


You see, when you use LSMASH.dll there are four functions inside:

LWLibavVideoSource() / LWLibavAudioSource()
LSMASHVideoSource() / LSMASHAudioSource()

now, the first two are what we would refer to as indexers, in fact they create a proper index file by seeking through the whole video and "indexing" the whole video (and audio).
This is exactly what FFMpegSource2() does with FFVideoSource() and FFAudioSource().
The other two, though, the LSMASH ones, are not.
They only work with ISO containers like MP4 and the reason is that they don't index.
You might call them indexers as well, but they really don't properly index, instead, they read the information inside the container and trust the container's index values.
This makes sense and allows users to actually decode the video faster than by using a "proper" indexer that re-creates the index from scratch.
You see, some containers have already an index inside them.
What indexers like LWLibav, FFVideo etc do is to ignore the internal index inside the container and re-create their own to play extremely safe.
What LSMASH does is to find the index inside the container and use it.
Of course, the index in a container can be broken, corrupted, wrong for all kind of reasons, so it's always better to really "index" a file with an indexer that properly re-creates it, but that doesn't mean that LSMASHVideoSource() is totally useless as sometimes the index in a container is right and can be trusted and this allows you to save a lot of time.

Just think about me indexing 1.3 TB worth of masterfiles through a 1 Gbit/s network. If I trust the index of the container and avoid to re-create it from scratch I'm gonna save a lot of time, so, as you can see, there are some scenarios in which it can be used.

Besides, the world is good 'cause it's various, so the more options we have to decode a file in Avisynth the better! ;)

[MysteryX]

Encoding GoPro 5K footage. Using StaxRip editor, with x265 it encodes fine, but with NVenc for hardware acceleration, it always freezes at about 99%. No warning or anything. When it freezes, it says "729/748" frames.

x265 encodes fine but gives a warning

vpy [FLAW]: error occurred while reading frame 747


In Avisynth, it encodes fine both with x265 and NVenc. (with LSMashSource)

Final test, using Vapoursynth with ffms2 source and NVenc. Worked flawlessly.

The problem is LSMashSource for VapourSynth.

[kedautinh12]

I think you need report to developer
https://github.com/AkarinVS/L-SMASH-Works/issues

[Katie Boundary]

Oh... poor LSMASH... :(


You see, when you use LSMASH.dll there are four functions inside:


I didn't say Lsmash.dll; I just said Lsmash. I thought the context would make it clear that I was referring to Lsmashvideosource and Lsmashaudiosource.

[FranceBB]

I didn't say Lsmash.dll; I just said Lsmash. I thought the context would make it clear that I was referring to Lsmashvideosource and Lsmashaudiosource.

Ok, granted.
Still, I gotta be fair, having an indexer that doesn't create an index file means that it's faster to open in AVS however I wouldn't take the risk on MP4 files from somewhere from the web 'cause bad things can happen like in this case. On the other hand, when I receive .mxf files of 1.3 TB as UHD Masterfiles it could be useful as it would save hours of indexing time (assuming that the index inside the container is correct).

So, in a nutshell, I'd like LSMASH to support MXF as well one day, but for anything that doesn't come from a professional studio, I wouldn't take any chances and I would use an indexer that indexes the file. :)

[MysteryX]

He did investigate the issue.
The video file is invalid in that the last two frames both contain the same
poc (26).
ffms2 ignores the error but lsmas is more picky.

ffms2 does work for GoPro videos, but in some cases, the frames in the source video don't match the frames in the output. Something gets skipped. That's probably due to the same issue.

Is there any way I can open GoPro videos without such glitches?

[videoh]

He did investigate the issue.

ffms2 does work for GoPro videos, but in some cases, the frames in the source video don't match the frames in the output. Something gets skipped. That's probably due to the same issue.

Is there any way I can open GoPro videos without such glitches? Any chance of a link to the video? I looked but couldn't find it.

[MysteryX]

Here's a sample video (https://mega.nz/file/TIoFya6Q#KQ3rZxzYbWAZxNG3eIOlZkf9-snH0Rki3LHwO8DvGjo)

I just converted a GoPro 5K HEVC video to UT Video with FFMPEG and it gave this list of errors:
- Using non-standard frame rate 29/1
- deprecated pixel format used, make sure you did set range correctly
- Duplicate POC in a sequence
- Error parsing NAL unit #0
- Error while decoding stream #0:0: Invalid data found when processing input

...

[videoh]

Seems to work fine with DGDecNV.

[FranceBB]

Who's the current maintainer of LSMASH and in which repository can I open an issue on Github about LWLibavAudioSource?
It used to be HollyWu but if I remember correctly last time he retired... so... who's handling this now?

[kedautinh12]

Just creat an issues and Asd-g will fix it

[FranceBB]

Just creat an issues and Asd-g will fix it

Yes, but I didn't know where, I mean in which repository xD

In the end, I figured it out and I opened it here:

https://github.com/AkarinVS/L-SMASH-Works/issues/17

I feel like I'm always the only one who comes up with Indexing bugs on weird audio codecs like last time with IPCM.
This time is DolbyE ehehehehehe

[kedautinh12]

Yes, but I didn't know where, I mean in which repository xD

In the end, I figured it out and I opened it here:

https://github.com/AkarinVS/L-SMASH-Works/issues/17

I feel like I'm always the only one who comes up with Indexing bugs on weird audio codecs like last time with IPCM.
This time is DolbyE ehehehehehe

It's for Vapoursynth and AkarinVS won't support avs, just creat issue here for avs
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[FranceBB]

Opened a new bug about a ZLog2 HDR H.265 UHD file (yes, there's more than just PQ and HLG in the world ehehehe): https://github.com/AkarinVS/L-SMASH-Works/issues/18

[kedautinh12]

L-SMASH-Works (20220406)
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[FranceBB]

L-SMASH-Works (20220406)
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

Uh, a new version, it's always nice when indexers are updated! :)

I have a question, or rather a feature request: when an .xml is given to the indexer like LWLibavVideoSource("CPL.xml") and LWLibavAudioSource("CPL.xml") please try to use the IMF decoder built into FFMpeg rather than return an error.
That would be really cool as this way we're gonna have IMF support inside Avisynth! :D

[kedautinh12]

You can ask in issues tab :D

[FranceBB]

You can ask in issues tab :D

Done. :)

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/16

[Emulgator]

Tiny cosmetics, don't remember when it sneaked in.
At first opening and indexing the resampler rate default seems to be uninitiated:
a=LWLibavAudioSource("F:\file.avi"
"LWLibavAudioSource: Failed to open resampler"
but indexes anyway.
Reloading after indexing throws no error.

[gispos]

Tiny cosmetics, don't remember when it sneaked in.
At first opening and indexing the resampler rate default seems to be uninitiated:
a=LWLibavAudioSource("F:\file.avi"
"LWLibavAudioSource: Failed to open resampler"
but indexes anyway.
Reloading after indexing throws no error.

I have the same problems when PCM audio stream is included in the video.
Have reported this here: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/20

[Emulgator]

Yes, confirmed: In my case PCM Codec ID "twos" Big Endian/Signed 16bps@48kHz wrapped in .MP4 from a Panasonic DC-GH5.

[kedautinh12]

L-SMASH-Works 20220505
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

If FranceBB rep issue quickly, we will have IMF support function :D

[FranceBB]

L-SMASH-Works 20220505
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

If FranceBB rep issue quickly, we will have IMF support function :D

Sorry, I've been a bit overwhelmed. Testing first thing tomorrow, I promise.

[LigH]

If FranceBB rep issue quickly, we will have IMF support function :D

Looking forward to that one:

This message will self-destruct in 5 seconds. Good luck, Jim.

https://static.wikia.nocookie.net/missionimpossiblefanon/images/9/90/Imf_logo_by_asainguy444-d6yhy6s-1.png

:sly:

[FranceBB]

Testing on a brand new content, the Sony UHD HDR PQ, RGB Full Range 12bit 4000 nits Remaster of Big Fish, the movie:

https://i.imgur.com/4DPzFAi.png
https://i.imgur.com/PRjylTP.png

Indexing is in progress, the .lwi file is being created and it's growing:

https://i.imgur.com/lgkAqXj.png

but it's gonna take a while I'm afraid:

https://i.imgur.com/hgHndS6.png
https://i.imgur.com/qDcw9zH.png

anyway, the fact that the indexing is in progress is indeed very promising.
I'll keep you posted, guys, thanks! :)

[kedautinh12]

Yeah 835 gb :D

[FranceBB]

Yeah 835 gb :D

Ladies and gentlemen: it works like a charm! :D

https://i.imgur.com/Ic3WacT.png

I replied to the ticket and closed it. https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/16
Very very very well done, guys!
I gotta be fair, LWLibav is my favorite Avisynth indexer 'cause you guys always picked my issues up and helped me (unlike *some other devs of another indexer which shall remain nameless*).
You did that with the DNX 185 frame slice issue, you did that with the IPCM audio decoding issue and now you've done it again with IMF support.
Thank you so much, I really mean it, whenever I had an issue, you've been there for me.

[kedautinh12]

Ladies and gentlemen: it works like a charm! :D

https://i.imgur.com/Ic3WacT.png

I replied to the ticket and closed it. https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/16
Very very very well done, guys!
I gotta be fair, LWLibav is my favorite Avisynth indexer 'cause you guys always picked my issues up and helped me (unlike *some other devs of another indexer which shall remain nameless*).
You did that with the DNX 185 frame slice issue, you did that with the IPCM audio decoding issue and now you've done it again with IMF support.
Thank you so much, I really mean it, whenever I had an issue, you've been there for me.

But i think you need rep quickly next time :D

[FranceBB]

But i think you need rep quickly next time :D

Yeah, definitely, but it's been a really busy week as we're migrating our Loudness Correction workflows from the old unsupported Dolby DP600 to Emotion, so I had to learn how to create workflow in there to create DolbyE (https://i.imgur.com/6LJ18Kl.png) and how to call their APIs (https://i.imgur.com/EqR0xSi.png) and make them work with our system etc etc etc
I wish I could spend all my time at work on Avisynth, but sadly I can't... :(

[mastrboy]

Is it possible to expose the "apply_grain" parameter for the dav1d decoder to avisynth in LSMASHVideoSource/LWLibavVideoSource? (https://videolan.videolan.me/dav1d/structDav1dSettings.html#a900e1cfb50a473fb1640f9af5a8faaa5)

[kedautinh12]

Ask Author's L-Smash works
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[flossy_cake]

Hello, is it possible to use DXVA2-copyback decoding with LWLIbavVideoSource? Or can it be configured to use LAV Video Decoder which supports DXVA2-copyback?

I tried prefer_hw, and I see there is decoder parameter also, but nothing in the wiki saying it supports DXVA2-copyback.

edit: also is it possible for LWLibavVideoSource to "signal downstream" so that the renderer (eg. MadVR) receives all the correct metadata like matrix, primaries, dynamic range etc. These all seem to go missing when I use Avisynth -- normally when I use LAV Splitter & Decoder without Avisynth it signals all those metadata to MadVR which helps with colour accuracy.

[flossy_cake]

Perhaps something like this?

LoadPlugin("LSMASHSource.dll")
LWLibavVideoSource("video.mkv")
Trim(0,-1)



Is it normal for cachefile creation to take 30 seconds for a 1 hour of video loading from SSD with an i7-4790k?

Each cache file is 16MB * 56 episodes = 896mb of cachefiles just for 1 series... :eek:

edit: tried making the cachefile from ffmpeg vs MPC-HC and the time is the same. Tried setting threads=n manually. CPU usage is around 4% during cachefile creation...bottleneck maybe?

[FranceBB]

Ok, since no one is replying, I'll answer a few questions.

1) Frame properties

Whenever an indexer indexes a source in a modern version of Avisynth+ frame properties are passed through and you can see them if you open the script in AVSPmod mod and you right click on the preview. Before those there were only clip properties, but now with frame properties everything an indexer indexes is passed through and can be used by third party programs receiving the uncompressed audio and video stream in RAM from the frameserver and use those, like it happens in ffmpeg. The fact that your final player/decoder opening the AVS doesn't get them means that it doesn't support frame properties, so it's not an LWLibavVideoSource() issue but rather your player, so you should ask them to support frame properties.

P.s beware that frame properties support is still experimental for most Avisynth filters etc


2) Indexing

Indexing can take a very long time and it's almost always a bandwidth problem. Honestly, 30 seconds for a 1h video sounds pretty damn fast to me! It's pretty clear that you're working on small file size videos, probably consumer stuff, so 30 seconds for 1h is pretty good/alright in terms of performance. Keep in mind that when I index UHD masterfiles over a 1 Gbit/s network it takes around 7 hours to index a 1h and 30 min movie (around 700-800 GB), so I'd say your performance is good. Last but not least, indexing files are larger for LWLibavVideoSource() and LWLibavAudioSource() than they are for FFVideoSource() and FFAudioSource(), so it's normal. Oh, almost forgot, one final note, if you happen to have like MP4 with H.264 and AAC etc or any other ISO format, you can use LSMASHVideoSource() and LSMASHAudioSource() as indexers in Avisynth but unlike normal indexers, they're gonna use the index already available inside the container to load the content straight away. Although this is supposed to be safe if the container is fine, it might lead to issues if there's something wrong with the container.

[flossy_cake]

Thanks

[Katie Boundary]

Is there an XP-compatible fork of this?

[FranceBB]

Is there an XP-compatible fork of this?

Unfortunately, no. :(
Not an updated one at least... :(
Last time LWLibav was compiled for XP, 16bit interleaved was still the default for high bit depth sources, so you can imagine how old it is...

[manolito]

The latest XP compatible LSmash Version I am aware of is this one:

lsmash-r784-150316-XP

It is from 2015, let me know if I should upload it...

[StainlessS]

lsmash-r784-150316-XP Here :- http://megui.tmebi.de/stable/

[flossy_cake]

Is the author perhaps interested in releasing a version without mandatory caching, for use by those who just use it to watch video in realtime and don't need frame accurate seek?

Or am I misunderstanding things and is frame accurate seek needed for filters to work properly -- and if so, which ones?

:thanks:

[FranceBB]

Is the author perhaps interested in releasing a version without mandatory caching, for use by those who just use it to watch video in realtime and don't need frame accurate seek?


You can't, unless you're willing to use the index inside the container and take it for good (which might not be).
So, if you take LSMASHVideoSource() and LSMASHAudioSource(), it will NOT index the file but rather use the index inside the container and take it for good. Not only this might not be, but some containers just don't have it like .ts to name one, therefore it's limited to ISO Standard containers only (like mp4), however if this is what you want to preview files etc, then go for it. ;)



Or am I misunderstanding things and is frame accurate seek needed for filters to work properly -- and if so, which ones?


You do need to have frame accurate indexing in any scenario, to be fair. Imagine this:

1) You index a source with let's say DirectoShowSource() using the LAV Filters.

2) You want to trim a commercial, so you use Trim(1231, 1554) and then you go on. Everything looks fine in AVSPmod.

3) You run the encode, only to find out that the commercial didn't really start nor end where you thought it was gonna be and you gotta start over.


This is why having frame accurate indexing is important.
It's also important if you want to hardsub subtitles 'cause when you send out low res proxies to the subtitling team and you get subs back, you want not just timecodes but frames to be accurate (especially if they have been doing typesetting on them). It's also important if you use ANY temporal filter, like MVTools to perform vector calculations or if you wanna perform like temporal denoise etc.

I mean, I can't imagine any reason not to use frame accurate indexing.

There are of course still cases that remain valid to use things like DirectShowSource(), for instance, some proprietary codecs like the Red Raw etc for which you legally installed the proprietary codec and you can leverage on DirectShowSource() in Windows to provide the decoded frames, but you can't use any open source indexer 'cause they wouldn't be able to index those files as they don't have decoders.

[flossy_cake]

Thanks I tried LSMASHVideoSource but like you said it's quite limited and none of my videos open with it.

I'm only using Avisynth for real time playback viewing of my video library. Real time decimation, deint, cropping, aspect, chroma shift etc. so I probably don't need frame accurate seeking.

If anyone knows of a way to disable caching/indexing please advise -- maybe there is some undocumented string for one of the cache params to forcefully disable it?

[FranceBB]

If anyone knows of a way to disable caching/indexing please advise -- maybe there is some undocumented string for one of the cache params to forcefully disable it?


You can't disable indexing unless you use DirectShowSource() + third party codecs like LAV.

About the "I'm using it only to play stuff in my library", why not using PotPlayer which supports Avisynth post-processing via an Avisynth Script?
PotPlayer will pipe to Avisynth automatically under the hood, thus avoiding indexing.

[StainlessS]

Thanks I tried LSMASHVideoSource but like you said it's quite limited and none of my videos open with it.

LSmashVideoSource is for ISO containers only, according to my list M2TS is not type ISO.

Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3gpp2"||s==".3g2"||s==".mj2"||s==".dvb"||
\ s==".dcf"||s==".m21"||s==".m4a"||s==".m4b"||s==".m4p"||s==".k3g"||s==".jpm"||s==".jpx"||s==".mqv"||s==".ismv"||s==".isma"||s==".f4v")}


https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format

https://en.wikipedia.org/wiki/Comparison_of_video_container_formats

EDIT: Sort of related, ftyp's :- http://www.ftyps.com/
Also see note 1 [what is an ftyp] (a sort of FourCC).

LSMashVideoSource() is just for ISO [ie MP4 and similar] files.
Otherwise use LWLibavVideoSource(). [An alternative video source in LSmash dll]

Same for the audio, LSMashAudioSource(), and LWLibavAudioSource().

[flossy_cake]

About the "I'm using it only to play stuff in my library", why not using PotPlayer which supports Avisynth post-processing via an Avisynth Script?
PotPlayer will pipe to Avisynth automatically under the hood, thus avoiding indexing.

Thanks for the tip. Currently I'm exploring the same kind of thing with MPC-HC using CrendKing's Avisynth filter (https://github.com/CrendKing/avisynth_filter). As far as I can tell it inserts Avisynth between LAV decoders and the renderer, so no indexing required either, but I'm having a couple of dealbreaking issues with it (see Issues tab in the Git).

[kedautinh12]

L-SMASH-Works 20221109
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[FranceBB]

L-SMASH-Works 20221109
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

What an important changelog!


AviSynth: added support for XYZ12LE format (represented as RGB48).

Thank you so so so so much Asd-g! :D
I've never felt so welcomed and cared for, honestly!
Every single time I raise an issue with this indexer, it's taken into consideration and addressed.

I love LWLibav :D



By the way, I tested it and it does indeed work.
While FFVideoSource() converts XYZ to YUV, LWLibavVideoSource() preserves the original XYZ inside the video, thus displaying it correctly and allowing us to work directly with XYZ! :D

https://i.imgur.com/h9gHQ4m.png

[kedautinh12]

@FranceBB, you can comment about XYZ in FFMpeg, i think author will update for it :D

[flossy_cake]

Here is a clip which LWlibavVideoSource cannot seem to play properly: https://drive.google.com/file/d/1lHB_EPFEKAQEiKab5LlYq0EyY9OOObeT/view?usp=share_link

It seems to have something to do with that soft telecine / repeat fields issue causing it to go out of sync with audio because the repeat field flags aren't being obeyed correctly (discussion about that back on page 58 (https://forum.doom9.org/showthread.php?t=167435&page=58)).


file = "480i multiple cadences.mkv"

AudioDub(LWlibavVideoSource(file, repeat=false), LWLibavAudioSource(file, stream_index=-1))

ScriptClip(last, """SubTitle(String(propGetAny(last, "_FieldBased")))""")


If I set repeat=true I get:

https://i.ibb.co/3yd8PbB/1.png

The _FieldBased frame property might offer a clue: the panning sequence starting at 1:25 is showing a _FieldBased value with pattern: 0-1-2, which means "progressive-bff-tff" which doesn't seem right for a 3:2 sequence, so maybe it has something to do with that?

FFMpegSource2 doesn't populate the _FieldBased prop, but can play it correctly with rffmode=1 which means "Honor all pulldown flags":


FFMpegSource2(file, rffmode=1, atrack=-1)


But, something is still not right: if I do simple bob deinterlacing with:


FFMpegSource2(file, rffmode=1, atrack=-1)
Bob()


The result is that "jumping back and forth" artefact which looks like the field order is wrong. So I try to correct that with:


FFMpegSource2(file, rffmode=1, atrack=-1)
AssumeTFF()
Bob()


^ this is better, but there are still some sequences which appear to have the wrong field order -- look at the "Series Story Editor" at 00:14, it still moves in the wrong direction for one frame before jumping forward 2 frames. I would have thought that was impossible with Bob().


edit: it seems these dynamic changes in field order need to be obeyed, in addition to the repeat field flags, in order to get correct playback. Still, it doesn't make sense to me as I thought analogue CRT would just scan field1 followed by field2 at all times. Seems my understanding is completely wrong: https://www.dvmp.co.uk/digital-video.htm

[kedautinh12]

I think you need report here to fix faster
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[flossy_cake]

I think you need report here to fix faster
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

Thanks, done.

[isidroco]

I have a problem with small lagarith clips made from pictures with repeated null frames: https://www.mediafire.com/file/hwdf8075reip6ta/07_-_creditos.avi/file
Using it to make a slideshow. DirectShowSource will add an extra frame. And LWLibavVideoSource will only return one frame instead of 150.
This is my test script:

nameS="07 - creditos"
extS=".avi"
# v= DirectShowSource( nameS + extS , ConvertFPS=true)
# v= trim(v, 1, length= framecount(v)-1 )
v= LWLibavVideoSource(nameS+extS)
v= ScriptClip(v, """
Subtitle(String( int(current_frame/30))+" seconds",
\ align=5, size=height(v)/16 )
""")
# v= info(v, size= height(v)*.05)
return v

[Hotte]

Video: AVDH 1280x720 29.97fps 97223kbps [V: Apple Video Media Handler [eng] (dnxhd dnxhr hqx, yuv422p10le, 1280x720, 97223 kb/s)]
Audio: PCM 48000Hz stereo 1536kbps [A: Apple Sound Media Handler [eng] (pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s)]

Codec ID: sowt

My guess would be that the endianness of the PCM audio was not correctly respected.

Hi, all my Videos containing pcm_s16be or pcm_s16le audio are fully distorted when read with LSmashAudioSource, while LSmashVideoSource is just fine. I've got the latest build from 20221109. It is a pity to wait for indexing of workaround solutions just to get audio right.

Any news about this ?

[kedautinh12]

Are you try LWLibavAudioSource??

[Hotte]

I did and it works. But it needs indexing, which I'd like to avoid and should not be necessary due to the fact that LSmashVideoSource works perfectly on my files for the video-stream without any indexing.

And: I do not understand why these files play instantly without issues and without any indexing if you open them directly with VDub2.

Something seems wrong with LSmashAudioSource and PCM-Audio from everyday-movs from everyday-cams like Nikon D5300 or Panasonic G9.

[kedautinh12]

You can create issue to fix faster or developer will explain about it
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[Hotte]

You can create issue to fix faster or developer will explain about it
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

Thanks. I opened issue #26

[StainlessS]

I do not understand why these files play instantly without issues and without any indexing if you open them directly with VDub2.
I think that is because the ISO container files [mp4 etc] that LSmashVideoSource() opens, already have some sort of index, as do AVI files.

EDIT: As posted in #1422,

Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3gpp2"||s==".3g2"||s==".mj2"||s==".dvb"||
\ s==".dcf"||s==".m21"||s==".m4a"||s==".m4b"||s==".m4p"||s==".k3g"||s==".jpm"||s==".jpx"||s==".mqv"||s==".ismv"||s==".isma"||s==".f4v")}

At least one of above is an audio-only ISO file. [".m4a"]

Only use LWLibavVideoSource() for non ISO video files. [attempt to open non ISO with LSmashVideoSource() will fail].

EDIT:
Here is a reasonably simple function to open files based on extension, works pretty well and could be used as an example to create your own,
https://forum.doom9.org/showthread.php?p=1874068#post1874068
Is implemented as an Avisynthesizer_Mod template.

See also the following posts where may have been improved.

And here:- https://forum.doom9.org/showthread.php?p=1880511#post1880511
Hello_Hello did his own version of a similar script.

Edit: Also, here description of my function (presumably the last mod of mine in that thread), where can change preferred order of source filters tried [EDIT: via Pref arg].
Well sometimes a source filter dont work proper, and is good to have an alternative, and way to force that alternative. [and 5 is better than 2]

Currently lookin' summick like this


Req:- RT_Stats, CallCmd, MIFO_Lib(MediaInfo script lib), MediaInfo CLI v0.7.83+ (may work with later versions)
Also, FFMS, LSmash, RawSourcePlus or RawSource (one of them), DSS2.

Function GetSeq(String "vFn",String "aFn"="",String "Pref"="",Int "VTrack"=-1,Int "ATrack"=-1,Val "fpsNum"=Undefined,Val "fpsDen"=Undefined,
\ Bool "Cache"=True,String "CacheDir"="",Bool "CacheDel"=False,Int "Threads"=0,Int "RFFMode"=0,String "TimeCodes"="",Int "AdjustDelay"=-1,String "VarPrefix"="",

vFn, Default "", default is NO video file. Can also use VTrack=-2 for no video. If "" and Vtrack != -2, then returns a single frame BlankClip, YV12 640x480, subtitle "No Video File".
aFn, Default "". Where "", will use audio from vFn. Can use ATrack=-2 for no audio.

Pref, Default "" {Same as "AIRFLD"). Order in which source filters are tried.
"A" = AviSource() (Only if extension is *.AVI, WavSource if Audio is WAV).
"I" = LSmashVideoSource() (Only if extension is an ISO type file, ie mov, mp4, m4v, 3gp, 3g2, mj2, dvb, dcf, m21.).
"R" = RawSourcePlus() (Only if extension is ".y4m".) RawSourcePlus if Avs+ and RawSource if Not Avs+.
"F" = FFVideoSource().
"L" = LWLibavVideoSource(), from LSMash.
"D" = DSS2()

VTrack, Default: -1 (auto. Video track number. If -2, disable video.
ATrack, Default: -1 (auto). Audio track number. If -2, disable audio.

Cache, Default True. False disables the writing of an index file (LWLibavSource and FFMS2 only).
CacheDir, Default Undefined. The directory location of the index file written by FFMS2 only. NOTE Directory NOT filename, No backslash at end of folder name.
If cache==true and CacheDir = "%TEMP%" then will be set to system environment user TEMP folder.
CacheDel, Default False [forced False if Cache=false] . If true, then auto delete FFMS[.ffindex] & LWLibavSource[.lwi] index files on clip closure.

Threads, Default 0(auto). The number of threads for LSMASH / FFMS4 to use.

RFFMode, Default 0. Honour repeat field flags (LWLibavSource & FFMS2 only).
FFMS2: 0=false, 1=true, 2=force film.
LWLibavSource: 0=false, 1=true, 2=false.


fpsnum,
fpsden, Defaults Undefined. Framerate for VFR->CFR conversion. # *** INGNORED if AVI ***
Specify FrameRate using fpsnum (Numerator) and fpsden (Denominator)
num only, eg fpsnum=25.0, FrameRate = 25.0 FPS
num & den, eg fpsnum=24000 fpsden=1001, FrameRate = 23.976 FPS (Both should be type Int).
neither specified FrameRate = UnDefined, ie whatever source filter thinks it is. (no VFR->CFR conversion)

Timecodes, Default "". FFMS only, write Matroska Timecodes to filename Timecodes.
AdjustDelay, Default -1. FFMS Only.
VarPrefix, Default "". FFMS only, Variable for FFInfo().

[Katie Boundary]

lsmash-r784-150316-XP Here :- http://megui.tmebi.de/stable/

Have I mentioned this year that you're my favorite person on Doom9? :)

[StainlessS]

It's lovely to be somebodies fav, cos everybody else hates me :(

but, seems hello_hello posted about the file first on the D9, [2017]
It appears to be missing for a 32bit operating system setup and dare I say it..... XP.

I think for the older L-Smash it should be msvcr100.dll.
There's a copy in the "lsmash-r784-150316-XP.7z" file but not in the "MeGUI\tools\lsmash" folder. I assume there should be?

I'm not having problems as I have the appropriate runtime installed, but I thought I'd mention it.

[real.finder]

It's lovely to be somebodies fav, cos everybody else hates me :(

but, seems hello_hello posted about the file first on the D9, [2017]

but I don't hate you :eek:

[StainlessS]

Not yet you dont, give it time. :devil:

EDIT: In reply to gispos next post::
Well you should, I'm nasty :devil:

[gispos]

I don't hate you either, my dear. :)

[FranceBB]

It's lovely to be somebodies fav, cos everybody else hates me :(

Nah, who could hate you?
You're like a lovely added grandpa for us ehehehehehe

[StainlessS]

I've had enough of you sychophants, now be quiet. :)

Sick-o-phants
https://www.cosgan.de/images/smilie/ekelig/n0203.gif https://www.cosgan.de/images/smilie/tiere/k015.gif

EDIT To Below, : And MXM 2U2 LigH.

https://www.cosgan.de/images/smilie/xmas/z700.gif

where is HanFrunz, bit late again.

[LigH]

MXM 2U2, Tinman.

[FranceBB]

For those who didn't get the GitHub notification, asd-g posted a new version here:

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

Changelog:
LSMASHAudioSource: fixed lsmash wrong endian box. (#26)
FFmpeg updated to e0cb89c.
l-smash updated to 642c4cd.

[LeXXuz]

Anyone else still having problems with special characters in the file path/name like German umlauts?

[nechto]

I have issues with Ru symbols ...

[kedautinh12]

Anyone else still having problems with special characters in the file path/name like German umlauts?

Me too with Vietnamese alphabet :D

[guest]

@ LeXXuz, nechto & kedautinh12...

check this out:-

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/31

and

https://superuser.com/questions/1715715/can-i-enable-unicode-utf-8-worldwide-support-in-windows-11-but-set-another-enco

This may "fix" the special characters problems...

But I'm having decoder problems.

UPDATE:- Setting the UTF-8 didn't fix my decoding problem, BUT, either of the "test" builds DID :). So now "we" just need a "fixed" L-Smash release. :)

[LeXXuz]

:thanks: I'll give that a spin. :)

[guest]

There's a new build :)

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/tag/20230507

[FranceBB]

Yep, I've got the notification from GitHub yesterday evening but I was watching the Coronation Concert on the BBC (the lights display with the drones was stunning as always) and I didn't feel like booting Windows ehehehehehe
Anyway, as far as Italian characters are concerned, looks like the UTF-8 issue is fixed.

https://i.imgur.com/tKqb9tR.png

I'll leave you guys testing on other esoteric languages :P

[kedautinh12]

L-SMASH-Works 20230611
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases


Fixed _EncodedFrameTop and _EncodedFrameTop, close #32. Changed behavior - they are showed only when there are frames to be repeated.
Fixed _FieldBased when repeat is not false.
Used libraries:

FFmpeg d78bffb.
l-smash 2c0696c.
obuparse 055be27.
zlib 1.2.13.
dav1d 1.2.1.
libxml2 v2.11.4.
nv-codec-headers n12.0.16.0.
xxHash v0.8.1.
mfx_dispatch 5a3f178.

[flossy_cake]

Unfortunately not fixed in latest version:

Here is a clip which LWlibavVideoSource cannot seem to play properly: https://drive.google.com/file/d/1lHB_EPFEKAQEiKab5LlYq0EyY9OOObeT/view?usp=share_link


file = "480i multiple cadences.mkv"

AudioDub(LWlibavVideoSource(file, repeat=true), LWLibavAudioSource(file, stream_index=-1))

ScriptClip(last, """SubTitle(String(propGetAny(last, "_FieldBased")))""")


https://i.ibb.co/3yd8PbB/1.png



However I have done rips of my own physical copy NTSC dvd's using MakeMKV and those work fine with repeat=true so I'm thinking it's some issue with the software that was used to create the MKV file. In all cases it's MPEG2 video stream remux.

Another test clip same problem: https://rationalqm.us/misc/lainvob.vob

[tebasuna51]

Please support AviSynth+ r4001 (https://forum.doom9.org/showthread.php?p=1989323#post1989323) including the audio channel mask.

[kedautinh12]

Report here will be faster
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[tebasuna51]

Report here will be faster


Done https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/35

[tebasuna51]

A new version to test in https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/35

For me work fine with some samples ac3, eac3 and dts.

[kedautinh12]

L-SMASH-Works 20230716
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

AviSynth: set audio property Channel Mask (#35).
Used libraries:

FFmpeg e62f344.
l-smash 2c0696c.
obuparse f40598f.
zlib 1.2.13.
dav1d 1.2.1.
libxml2 v2.11.4.
nv-codec-headers n12.0.16.0.
xxHash v0.8.1.
mfx_dispatch 5a3f178.

[almosely]

Just wanted to report that I have updated from L-SMASH-Works (2022-11-09) to the newest and it did not work for me anymore. I tried to play a youtube video from 2017 ...

LWLibavVideoSource("Federer.vs.Nadal.720p.HD.Australian.Open.Final.2017.mkv", format="YUV420P8", prefer_hw=1)

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 7 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 7 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 3 h 18 min
Bit rate : 3 778 kb/s
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 59.940 (60000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.068
Stream size : 5.24 GiB (96%)
Default : Yes
Forced : No

... but it stuttered and had pixel errors all over. Hardware is GeForce GTX 660 TI and Core i7-3770. Windows 10.

Then I tried backwards until L-SMASH-Works (2023-05-07) is working for me again. So 06 and 07 aren't working anymore. I am using the newest AVS+ 64 and my C++ Runtimes are up to date too. Just my 2ct.

[kedautinh12]

Report here will be faster
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[FranceBB]

Also keep in mind that hardware accelerated decoding has always been a hit and miss.
If you want something reliable, software decoding is the way to go (i.e don't specify prefer_hw=1).

[almosely]

Never had any problems like that with L-Smash or DGDecNV in the last 10 years - so no hit-n-miss for me. Encoding without hw-acc-decoding is much slower.

[FranceBB]

Wow, I'm very surprised to see that it's actually stable even with videos with ref larger than 4.
Anyway, as kedautinh12 rightly pointed out, asd-g is the one keeping the development ongoing for LWLibav.
He generally listens to user, but he prefers to keep track of issues through GitHub rather than through posts on Doom9.
Please report the regression on GitHub by opening an issue here: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues
I'm sure he'll take a look at it. :)

[almosely]

Already done before your two posts ;-) And he looked at it.

[Atak_Snajpera]

Never had any problems like that with L-Smash or DGDecNV in the last 10 years - so no hit-n-miss for me. Encoding without hw-acc-decoding is much slower.

Have you done some real tests?
Encoding process for example x265 will consume ~95% of CPU time while decoding rest ~5%.

I wouldn't say that extra 5% will give you much faster encoding.

[almosely]

Real tests? I am encoding almost daily for over 20 years. Of course I am talking out of experience. With AVS I started with xvid, then changed to x264 and since a few years to x265. A few days ago I was encoding a 1080p video (usually I encode 720p). With software decoding: 6 fps. With hardware decoding 9 fps. So 50% faster in that usecase ...

[Atak_Snajpera]

So you encoded exactly the same video file twice right?

[almosely]

So, why is that you think everybody else is doing it wrong? ...

[almosely]

So, I did some tests, even reverted to earlier AVS and plugin-versions from that encoding-time, but sadly I cannot reproduce that penomenon anymore. I do not have that particular file anymore.

But I do remember, I started an encode with hw-decoding on (as usual). The estimated encoding time has been about 8 hours and the fps had been around 9. That did not change after having a look at that estimation a few hours later. The next day that encode ended somewhere after approx. 6 hours with an encoder crash. That happens sometimes, rarely, and usually changing a bit of the parameters of requestlinear solves that issue. So I wanted to get that encode finished correctly but did not want to risk another crash. That's why I gave hw-decoding switched off a try. I did not change anything else. Then the estimated encoding time was something about 12 hours and the fps were about 6 and that was true after the job finished - it crashed too, but after approx. 11 hours. So, hw on/off wasn't the reason fot that. Then I had to start a third encode for the last minutes to glue the peaces together in the end. Those crashes started with AVS+ (when prefetch(x,y) was introduced) and I have no idea what's the reason for that. There wasn't running anything in the background that time.

And I remember that I made some tests some years ago with hw-dec on/off and everytime hw-dec on has been faster, a bit, not as much as with that particular file, but faster, without any cost in power (watts). So I use hw-dec on every time.

[Atak_Snajpera]

So, why is that you think everybody else is doing it wrong? ...

Placebo effect

source AVC blu-ray 1080p (no avisynth filters!)
https://i.postimg.cc/RVtMFv0R/Untitled-1.png

source HEVC HDR10 UHD 2160p (no avisynth filters!)
https://i.postimg.cc/W158DFXL/Untitled-2.png

If you use software encoder x265 (with default medium preset) bottleneck is so huge that hardware decoding does not help.
Basically decoder most of time is sitting idle and waiting for requests from encoding process.

In my case decoding process was using less than 5% of CPU time.

I really do not care about that few percent and prefer much stable CPU decoding.

[almosely]

Come on ... You are one person out of billions. The world is bigger than your imagination and experience. What is the meaning of compairing completely different hardware, yours and mine?

I am not using any default presets.

"In my case" and "I really do not care about" is the whole point of this.

I just wanted to report an error within the last two updates, no more.

[Atak_Snajpera]

That's why I asked if you did a REAL tests. Your claim that hardware decoding gives huge speed boost in encoding time is a lie.
If you do some filtering for example denoising/tonemapping in avisynth then you won't even get those 5% because even more time will be used by your cpu.
Hardware decoding is a placebo effect in this case. Deal with it.

[almosely]

You are rude, disrespectful, short-sighted, conceited and unreasonable. I won't exchange another word with you.

[LigH]

Nevertheless, he is right. Compared to the efforts of a video encoder, decoding with or with out GPU support is a negligible difference.

Run your scripts through AVSMeter, compare these "benchmark" durations with the encoding duration: That ratio should tell you where the real bottleneck is.

[real.finder]

even if Atak_Snajpera is right, I think it's better to have both option just in case, also I think hardware decoder can help with weak cpu cases like laptop or low end or old hardware

[guest]

You are rude, disrespectful, short-sighted, conceited and unreasonable. I won't exchange another word with you.

So now we have some other opinions...

Either way, Atak has the knack of coming across like that, so props for you saying so, not many do !!!

[kedautinh12]

So now we have some other opinions...

Either way, Atak has the knack of coming across like that, so props for you saying so, not many do !!!

Long time to see, old man :D

[guest]

Long time to see, old man :D

How did you find me :D

Yep, keeping an Ultra-low profile these days, kinda lost interest :(

You're not posting as often as you used to, either.

[tebasuna51]

You are rude, disrespectful, short-sighted, conceited and unreasonable. I won't exchange another word with you.

Please guys remember the forum rule:
4) Be nice to each other and respect the moderator. Profanity and insults will not be tolerated. If you have a problem with another member turn to the respective moderator and if the moderator can't help you send a private message to Doom9.

There are 2 questions here:
1) Save time using GPU decode? A little but yes.
2) Last L-SMASH-Works-20230716 have a bug? I think so, then almosely is right and we can say thanks for the report.

1) I have a cheap Nvidia GeForce GT 1030 and running AvsMeter you can see the GPU decode is slow than with my CPU:
AviSynth+ 3.7.3 (r4003, 3.7, x86_64) (3.7.3.0)
Over "Roger Federer v Rafael Nadal Full Match Australian Open 2017 Final.mp4":
Number of frames: 304982
Length (hh:mm:ss.ms): 03:23:19.280
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25/1)
Colorspace: i420
06/07/2023 NVIDIA GeForce GT1030
FFVideoSource DgSource 249
-------------------- ---------------------
Frames processed: 304982 (0 - 304981) 304982 (0 - 304981)
FPS (min | max | average): 281.3 | 1807 | 960.5 149.9 | 489.9 | 474.5
Process memory usage (max): 190 MiB 335 MiB
Thread count: 25 17
CPU usage (average): 74.5% 8.6%

GPU usage (average): 2% 57%
VPU usage (average): 0% 74%
GPU memory usage: 402 MiB 459 MiB
GPU Power Consumption (average): 13.6 W 15.9 W

Time (elapsed): 00:05:17.510 00:10:42.809

But when I recode the file with x264 (crf 20) the DgSource avs is fast than FFVideoSource with the same stats (see log files):

DgSource encoded 304982 frames, 59.71 fps, 4513.20 kb/s (5107 sec)
FFVideoSource encoded 304982 frames, 57.89 fps, 4513.20 kb/s (5269 sec)
It is only 2:42 over 1:27:49 encode but maybe with a better GPU can go until the 5:17 used by the CPU decoder.

2) But when I test L-SMASH-Works-20230716 I see than there are something wrong:
LSMASHVideoSource 15/07/2023
----------------------------------
prefer_hw=0 prefer_hw=1
--------------------- ---------------------
Frames processed: 304982 (0 - 304981) 304982 (0 - 304981)
FPS (min | max | average): 45.87 | 2711 | 89.62 26.73 | 580.8 | 80.34
Process memory usage (max): 236 MiB 225 MiB
Thread count: 26 16
CPU usage (average): 19.6% 7.6%

GPU usage (average): 2% 15%
VPU usage (average): 0% 23%
GPU memory usage: 373 MiB 399 MiB
GPU Power Consumption (average): 13.7 W 14.2 W

Time (elapsed): 00:56:42.960 01:03:16.199
The full encode (prefer_hw=1) have different stats (see the log) and:
LSMASHVideoSource encoded 304982 frames, 44.84 fps, 5384.67 kb/s (6801 sec, 1:53:21)
And is wrong (https://www.sendspace.com/file/hfuk86), the prefer_hw=0 play fine and with same stats than both before(slow than DgSource).

[kedautinh12]

Try test ver
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/36#issuecomment-1665742509

[tebasuna51]

20230611 version is also wrong
20230507 work fine with prefer_hw=1
20230804 test version work fine now

[guest]

Try test ver
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/36#issuecomment-1665742509

Nice :)

And now there's a newer "test" build available.

[guest]

FYI @ tebasuna51

Originally Posted by almosely

You are rude, disrespectful, short-sighted, conceited and unreasonable. I won't exchange another word with you.

Please guys remember the forum rule:

4) Be nice to each other and respect the moderator. Profanity and insults will not be tolerated. If you have a problem with another member turn to the respective moderator and if the moderator can't help you send a private message to Doom9.

Thats all well & good, but there a handful of members that can come across as being rude, just like @ almosely mentioned, and generally nobody says anything.

So, I'm on his side, and the member targeted, is one of the rude ones.

I've had first hand experience.

[Selur]

Is there a separate .exe that could create the .lwi cache file?

[qyot27]

Is there a separate .exe that could create the .lwi cache file?
One exists in this patch:
https://github.com/afontenot/L-SMASH-Works/commit/522f5575e775deda382a3709883a3e85b13d8331

It hasn't been submitted upstream, though.

[kedautinh12]

L-SMASH-Works 20230806 1129.0.1.0
#36 - reverted HomeOfAviSynthPlusEvolution/FFmpeg@402d98c

Used libraries:

FFmpeg e521879.
l-smash 2c0696c.
obuparse f40598f.
zlib 1.2.13.
dav1d 1.2.1.
libxml2 v2.11.4.
nv-codec-headers n12.0.16.0.
xxHash v0.8.1.
mfx_dispatch 5a3f178.
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/tag/1129.0.1.0

[tebasuna51]

I make a test with that last version L-SMASH-Works 20230806 1129.0.1.0 and the same file in my previous post (https://forum.doom9.org/showthread.php?p=1990413#post1990413)

The x264 stats and bitrate are the same than DgSource or FFVideoSource using prefer_hw=1 or not, but are slow:
DgSource("C:\tmp\Test.dgi") encoded with x264 -crf 20 at 59,71 fps 5107,720 s. = 1:25:07.720
FFVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57,89 fps 5268,301 s. = 1:27:48.301
LSMASHVideoSource("..", prefer_hw=1) encoded with x264 -crf 20 at 49.06 fps 6216,510 s. = 1:43:36.510
LSMASHVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 48.92 fps 6234,300 s. = 1:43:54.300


And using AvsMeter the decoding time is still the same than show in point 2) of my previous post, near 1 h. when DgSource/FFVideoSource are 10/5 m.
At beginning AvsMeter show values like DgSource, with hw=1, or FFVideoSource, but after the CPU/GPU usage go to low values and finish with that big time.
Maybe someone can test this behaviour.

[StvG]

I make a test with that last version L-SMASH-Works 20230806 1129.0.1.0 and the same file in my previous post (https://forum.doom9.org/showthread.php?p=1990413#post1990413)

The x264 stats and bitrate are the same than DgSource or FFVideoSource using prefer_hw=1 or not, but are slow:
DgSource("C:\tmp\Test.dgi") encoded with x264 -crf 20 at 59,71 fps 5107,720 s. = 1:25:07.720
FFVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57,89 fps 5268,301 s. = 1:27:48.301
LSMASHVideoSource("..", prefer_hw=1) encoded with x264 -crf 20 at 49.06 fps 6216,510 s. = 1:43:36.510
LSMASHVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 48.92 fps 6234,300 s. = 1:43:54.300


And using AvsMeter the decoding time is still the same than show in point 2) of my previous post, near 1 h. when DgSource/FFVideoSource are 10/5 m.
At beginning AvsMeter show values like DgSource, with hw=1, or FFVideoSource, but after the CPU/GPU usage go to low values and finish with that big time.
Maybe someone can test this behaviour.

LSMASHVideoSource is the only function from the listed that doesn't create index.

What about testing LWLibavVideoSource?

[tebasuna51]

Despite the recommended method LSMASHVideoSource to open a .mp4 seems don't work better.
Using LWLibavVideoSource, and after a few seconds (<10) creating the .lwi, is the faster method with same stats and bitrate than others:
DgSource("C:\tmp\Test.dgi") encoded with x264 -crf 20 at 59.71 fps 5107,720 s. = 1:25:07.720
FFVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57.89 fps 5268,301 s. = 1:27:48.301
LWLibavVideoSource("..", prefer_hw=1) encoded with x264 -crf 20 at 60.11 fps 5073,731 s. = 1:24:33.731
LWLibavVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57.93 fps 5264,664 s. = 1:27:44.664

Fast using GPU or only CPU, the problem with AvsMeter is gone and is fast decoding with GPU:

LWLibavVideoSource
----------------------------------
prefer_hw=1 prefer_hw=0
--------------------- ---------------------
304982 (0 - 304981) 304982 (0 - 304981)
126.0 | 615.7 | 577.0 247.8 | 2420 | 958.8
168 MiB 213 MiB
16 26
7.0% 76.7%

65% 1%
88% 0%
756 MiB 664 MiB
16.3 W 13.7 W

00:08:48.532 00:05:18.073

For me LSMASHVideoSource can't be recommended at all.

[StvG]

Despite the recommended method LSMASHVideoSource to open a .mp4 seems don't work better.
Using LWLibavVideoSource, and after a few seconds (<10) creating the .lwi, is the faster method with same stats and bitrate than others:
DgSource("C:\tmp\Test.dgi") encoded with x264 -crf 20 at 59.71 fps 5107,720 s. = 1:25:07.720
FFVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57.89 fps 5268,301 s. = 1:27:48.301
LWLibavVideoSource("..", prefer_hw=1) encoded with x264 -crf 20 at 60.11 fps 5073,731 s. = 1:24:33.731
LWLibavVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57.93 fps 5264,664 s. = 1:27:44.664

Fast using GPU or only CPU, the problem with AvsMeter is gone and is fast decoding with GPU:

LWLibavVideoSource
----------------------------------
prefer_hw=1 prefer_hw=0
--------------------- ---------------------
304982 (0 - 304981) 304982 (0 - 304981)
126.0 | 615.7 | 577.0 247.8 | 2420 | 958.8
168 MiB 213 MiB
16 26
7.0% 76.7%

65% 1%
88% 0%
756 MiB 664 MiB
16.3 W 13.7 W

00:08:48.532 00:05:18.073

For me LSMASHVideoSource can't be recommended at all.

Thanks for sharing. What's the used ffms2 version?

[tebasuna51]

Your ffms2_r1363 (https://forum.doom9.org/showthread.php?p=1989259#post1989259) 06/07/2023

BTW the differences between
FFVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57.89 fps 5268,301 s. = 1:27:48.301
LWLibavVideoSource("C:\tmp\Test.mp4") encoded with x264 -crf 20 at 57.93 fps 5264,664 s. = 1:27:44.664
aren't important and can be for other run process (I try run alone but not all can be controlled)

The AvsMeter is best (5:17.510) for FFVideoSource, (5:18.073) LWLibavVideoSource.

The best, only a few, option seems LWLibavVideoSource("..", prefer_hw=1) with a good GPU.

[kedautinh12]

L-SMASH-Works 20231003 1141.0.0.0
Added support for BGR0.
The last requested frame is used to set _FieldBased (repeat=true).
Added frame property _AbsoluteTime.
Used libraries:

FFmpeg b4f9170.
l-smash 2c0696c.
obuparse f40598f.
zlib 1.3.
dav1d 1.2.1.
libxml2 v2.11.5.
nv-codec-headers n12.0.16.0.
xxHash v0.8.2.
mfx_dispatch 5a3f178.
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[FranceBB]

Thank you asd-g, as always, for everything.
Over the years LWLibav has become my de facto indexer for pretty much everything.
The amount of care you put towards fixing the issues that are reported and implementing the features that are requested has been hugely appreciated.
Thank you on behalf of all my colleagues too! :)

[kedautinh12]

L-SMASH-Works 20231010 1144.0.0.0
Fixed _AbsoluteTime start_time.
Fixed raw bitstreams fps (FFmpeg new API).
Used libraries:

FFmpeg b4f9170.
l-smash 2c0696c.
obuparse f40598f.
zlib 1.3.
dav1d 1.2.1.
libxml2 v2.11.5.
nv-codec-headers n12.0.16.0.
xxHash v0.8.2.
mfx_dispatch 5a3f178.
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[kedautinh12]

L-SMASH-Works 20231106 1147.0.0.0
AviSynth: reverted harmful commits about UTF-8 filenames.
Used libraries:

FFmpeg 2c85a07.
l-smash 2c0696c.
obuparse f40598f.
zlib 1.3.
dav1d 1.3.0.
libxml2 v2.11.5.
nv-codec-headers n12.1.14.0.
xxHash v0.8.2.
mfx_dispatch f6aac45.
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[FranceBB]

The special character names tackling shenanigan continues... xD
No but jokes aside, hopefully this is gonna be sorted once and for all.
As to the other part of the update, thank you asd as always.
I just pulled it to my test bench and I'll be testing it out after my regular cappuccino. :)

[kedautinh12]

L-SMASH-Works 20231117 1156.0.0.0
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[flossy_cake]

Thank you asd-g, as always, for everything.
Over the years LWLibav has become my de facto indexer for pretty much everything.
The amount of care you put towards fixing the issues that are reported and implementing the features that are requested has been hugely appreciated.
Thank you on behalf of all my colleagues too! :)

Yes I would like to echo this sentiment. Without LWLibav I don't think Avisynth would be much use for my growing DVD library. LWLibav is the only filter that seems to understands MPEG2 properly, particularly the repeat field flags and field order. It also seems highly compatible with 264 and 265, never had an issue with those, and supports 10-bit pixel formats too. Seek times are nice and quick too. I've read it's "frame accurate" too, which I don't actually know the meaning of -- does it mean current_frame can be relied on? So let's say you're trying to count frames after a scenechange or something, you can rely on knowing when you are exactly, say, 3 frames after a scenechange? I've not had any issues with this so I'm guessing it's working as intended.

The only negative thing I could say about LWLibav is the large file size of the .lwi cache files and the time it takes to generate them. I would very much like to see multithreading of the .lwi cache file creation process. That would personally benefit me the most when screening DVD's on HTPC to eliminate the sometimes lengthy wait times before video playback begins.

[FranceBB]

does it mean current_frame can be relied on? So let's say you're trying to count frames after a scenechange or something, you can rely on knowing when you are exactly, say, 3 frames after a scenechange?


That's exactly right. It won't miss frames while decoding so if you move to frame 3325, let's say, you can be absolutely confident that every time you move there it will always be frame 3325. ;)

[flossy_cake]

Don't forget about DGIndex and DGIndexNV.

Ah yes, my apologies to Donald Graft :o

From what I can tell though the required d2v index file cannot be created within an Avisynth script, so unfortunately that is a dealbreaker for me. It looks like maybe the DGIndexNV one can, but that is limited to NVidia GPUs which is also a dealbreaker even though I use an Nvidia GPU, I need my scripts to work on more than just NVidia systems.

[StvG]

... LWLibav is the only filter that seems to understands MPEG2 properly, particularly the repeat field flags and field order...

If your video is in mkv this ffms2 (https://forum.doom9.org/showthread.php?p=1993933#post1993933) is ok with MPEG2. It also adjusts the field frame prop/getparity per frame.

[flossy_cake]

From what I can tell though the required d2v index file cannot be created within an Avisynth script

I stand corrected - it looks like RT_Call (http://avisynth.nl/index.php/RT_Stats#Utility_Functions) and DGIndex CLI (https://www.rationalqm.us/dgmpgdec/DGIndexManual.html#AppendixB) may be able to achieve that.

Assuming RT_Call waits until the command is complete before allowing the next Avisynth filter in the chain (MPEG2Source) to be called - can you clarify StainlessS? @StainlessS :thanks:

[StainlessS]

Flossy, no, but see CallCmd(Synchronous=7), [EDIT: Using Open command]
https://forum.doom9.org/showthread.php?t=166063


CallCmd(clip,string "Command"="", string "Frames"="",string "Insert"="",int "Digits"=6,bool "Once"=true,int "Offset"=0,
bool "Hide"=false,bool "Debug"=false,string "Open"="",string "Close"="",int "Synchronous"=7)

Synchronous (default 7, range 0 - 7)
The Synchronous arg is a group of 3 bitflags,
bit 0 if set means wait for process to terminate before returning from the 'Open' command.
bit 1 if set means wait for process to terminate before returning from the 'Frames' commands.
bit 2 if set means wait for process to terminate before returning from the 'Close' command.
0 = All Asynchronous, ie dont wait at all.
1 = Wait only on 'Open' command.
2 = Wait only on 'Frames' commands.
3 = Wait on 'Open' and 'Frames' commands.
4 = Wait on 'Close' command.
5 = Wait on 'Open' and 'Close' commands.
6 = Wait on 'Frames' and 'Close' commands.
7 = Wait on ALL commands (default).

Beware, if you dont wait it is very possible that any future reliance upon output of command may fail due to the
output file being in use. Also, if debug true and waiting for process to complete, then will also show the process return
code on successful process creation (and completion) in DebugView, cannot do this if NOT waiting completion.


EDIT: OOps, actually, I think not, you have to provide already Opened source clip, and CallCmd command executed on first frame
during frameserve stage. [EDIT: Actually on CallCmd filter construction, during filter graph creation and BEFORE frameserving starts]
Maybe you could call eg RT_Sleep(10.0), and pole for existing index file, else loop.
EDIT: If using DGIndex Template, then created AVS file is created last after indexing. [Using DGIndex GUI, dont know about CMD line ver$]

EDIT: Gone t' pub.

[kedautinh12]

Wow, sound good :D

[flossy_cake]

Here's a test version for auto-indexing. This is for 64-bit DGDecode.dll (MPEG2). Currently Avisynth only. Make sure that the included DGIndex.exe is in the same directory as DGDecode.dll. The revised DGIndex.exe's only change is to print a message in CLI mode saying that indexing is taking place. Please advise your test results.

https://rationalqm.us/misc/DGDecode64_test.rar

We should take discussion away from this thread. How about here:

https://www.rationalqm.us/board/viewtopic.php?f=7&p=18807#p18807

Thanks for this. I couldn't get it to create d2v from within an Avisynth script, only from its GUI. I'll reply on your forum...

[kedautinh12]

I think you can create a new thread about DGDecode on doom9 forum

[tebasuna51]

WARNING: the sintax in last L-SMASH-Works-r1156.0.0.0 have a important change:

the recommended parameter for LWLibavAudioSource/LSMASHAudioSource(..., drc_scale=0,...) not work now, must be:
(..., ff_options = "drc_scale=0", ...)

[StainlessS]

I stand corrected - it looks like RT_Call (http://avisynth.nl/index.php/RT_Stats#Utility_Functions) and DGIndex CLI (https://www.rationalqm.us/dgmpgdec/DGIndexManual.html#AppendixB) may be able to achieve that.

Assuming RT_Call waits until the command is complete before allowing the next Avisynth filter in the chain (MPEG2Source) to be called - can you clarify

Flossy, no, but see CallCmd(Synchronous=7), [EDIT: Using Open command]
https://forum.doom9.org/showthread.php?t=166063


CallCmd(clip,string "Command"="", string "Frames"="",string "Insert"="",int "Digits"=6,bool "Once"=true,int "Offset"=0,
bool "Hide"=false,bool "Debug"=false,string "Open"="",string "Close"="",int "Synchronous"=7)

Synchronous (default 7, range 0 - 7)
The Synchronous arg is a group of 3 bitflags,
bit 0 if set means wait for process to terminate before returning from the 'Open' command.
bit 1 if set means wait for process to terminate before returning from the 'Frames' commands.
bit 2 if set means wait for process to terminate before returning from the 'Close' command.
0 = All Asynchronous, ie dont wait at all.
1 = Wait only on 'Open' command.
2 = Wait only on 'Frames' commands.
3 = Wait on 'Open' and 'Frames' commands.
4 = Wait on 'Close' command.
5 = Wait on 'Open' and 'Close' commands.
6 = Wait on 'Frames' and 'Close' commands.
7 = Wait on ALL commands (default).

Beware, if you dont wait it is very possible that any future reliance upon output of command may fail due to the
output file being in use. Also, if debug true and waiting for process to complete, then will also show the process return
code on successful process creation (and completion) in DebugView, cannot do this if NOT waiting completion.


EDIT: OOps, actually, I think not, you have to provide already Opened source clip, and CallCmd command executed on first frame
during frameserve stage. [EDIT: Actually on CallCmd filter construction, during filter graph creation and BEFORE frameserving starts]
Maybe you could call eg RT_Sleep(10.0), and pole for existing index file, else loop.
EDIT: If using DGIndex Template, then created AVS file is created last after indexing. [Using DGIndex GUI, dont know about CMD line ver$]

EDIT: Gone t' pub.

You could perhaps use CallCmd() to execute Open command on dummy clip eg BlankClip(length=1).CallCmd(Open="...",Syncronous=7), to create an index for the actual target file,
then just use index as per normal. [object being to use Callcmd with Syncronous=7 to wait until target file index is actually created]
[Open="..." being the command line to call the indexer on target file. Also, see BLUE EDIT line in above quote]

But it looks like the problem is solved anyway.

[flossy_cake]

But it looks like the problem is solved anyway.

Yes, DG released auto indexer for DGDecode but it's not compatible with mkv's (I remuxed all my physical copy DVD's to mkv using MakeMKV and plan to continue using that application for various reasons).

FFMS2 is also a good option now too, it seems. So I'm quite spoiled for choice when it comes to accurate DVD playback, and DGDecode is still nice to have as a backup option (just need to give it .vob files from eg. DVD Decrypter or DVD Shrink).

[flossy_cake]

In the interest of improving LWLibAV compability, here is an mkv which FFMpegSource2 can play but LWLibAV cannot:

https://drive.google.com/u/0/uc?id=1Pm2V6bbrrcTJezbta7HWjW3VEa5GBz7E&export=download

It is MPEG2 video remuxed from DVD using MakeMKV.


FFVideoSource("C:\S01E01.mkv", rffmode=1) # works
LWLibavVideoSource("C:\S01E01.mkv", repeat=true) # doesn't work


https://i.ibb.co/TbvG14J/debug-avs-snapshot-00-01-500.png

Curiously, if I trim 2 seconds off the end of it with:


"c:\program files\ffmpeg\bin\ffmpeg.exe" -ss 00:00:00 -to 00:23:52 -i "C:\S01E01.mkv" -c copy -map 0 "C:\S01E01_trimmed.mkv"


the file then plays fine with LWLibAV.

So the offending data appears to be within the last couple of seconds of the clip - hopefully that offers a clue as to why LWLibAV doesn't like it.

It may very well be that the last 2 seconds is somehow incorrectly encoded and FFMS2 should refuse to play it as well, but that is a decision for the authors of the source filters to decide how "fault tolerant" they should be.

[kedautinh12]

You neeed create issue here
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

[flossy_cake]

You neeed create issue here
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

Done, thanks.

[StainlessS]

WARNING: the sintax in last L-SMASH-Works-r1156.0.0.0 have a important change:

the recommended parameter for LWLibavAudioSource/LSMASHAudioSource(..., drc_scale=0,...) not work now, must be:
(..., ff_options = "drc_scale=0", ...)

Thanks Teb,
I've just myself discovered that problem and was about to report it.

I guess that I've got a lot of scripts to change,
would have been nice if old args could remain as was,
and any changes coped with within the LSmash dll,
ie create dll internal ff_options string from eg LSmashAudioSource(..., drc_scale=0).
It should be fairly easily 'do-able' -

EDIT: From the doc file,

LSMASHVideoSource(string source, int track = 0, int threads = 0, int seek_mode = 0, int seek_threshold = 10,
bool dr = false, int fpsnum = 0, int fpsden = 1, string format = "", string decoder = "",
int prefer_hw = 0, int ff_loglevel = 0, string ff_options = "")

# ...

+ ff_options (default : "")
Set the decoder options in FFmpeg.
The format is `key=value` separated by " ". (e.g. "drc_scale=0 auto_convert=0").

and for all other video audio filters, eg


LSMASHAudioSource(string source, int track = 0, bool skip_priming = true, string layout = "", int rate = 0,
string decoder = "", int ff_loglevel = 0, string ff_options = "")

# ...

+ ff_options (defalut: "")
Same as 'ff_options' of LSMASHVideoSource().



EDIT: I guess that if there are likely to be many more ffmpeg options added, then would be easiest to just leave it as is,
and change all existing scripts instead :eek:

[flossy_cake]

Why not have both? The drc_scale param can be removed from documentation so that new users start using ff_options, while old scripts maintain forwards compatibility.

[flossy_cake]

Oh it's just been added back: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/51

If `ff_options="drc_scale=x"` is used, `drc_scale` is ignored.

That's perfect imo.

[Selur]

Oh it's just been added back
but the build action failed: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/actions/runs/7154104282

[kedautinh12]

He will release if it's ready. He need fix some issues from Selur and flossy_cake

[flossy_cake]

I wasn't even aware drc_scale was supported as it isn't mentioned at the avisynth.nl page (only in readme.md on github).

I am interested in using it, but can't really test it out as I can't seem to find a filter or other tool that can report on whether the AC-3 track has DRC encoding. Can anyone suggest a tool for getting DRC info from video files with ac-3 tracks? MediaInfo doesn't seem to say, I think possibly as it's not flagged as metadata but is somehow baked into the packets inside the audio stream.

[tebasuna51]

... other tool that can report on whether the AC-3 track has DRC encoding...

You need extract the tracks and use LeeAudBi (https://forum.doom9.org/showthread.php?p=424067#post424067) to read all the frames (MediaInfo can't read all the frames)

The metadata are in the Audio Block, a value for each block, normally a frame have 6 blocks

File: test.ac3
Size: 961536 bytes
----------------------------------------- First Frame Info
SampleRate ..................: 0 (48000 Hz)
BitRate .....................: 14 (384 Kb/s)
FrameSize....................: 1536 bytes
...
Dynamic Range gain ..........: 5.89 dB (first block)
----------------------------------------- Revised full AC3 Info
Dyn. Range min/max : -11.51/5.89 dB
...
------------------------------------------------- End Info

[flossy_cake]

You need extract the tracks and use LeeAudBi (https://forum.doom9.org/showthread.php?p=424067#post424067)

:thanks:

[kedautinh12]

L-SMASH-works 20231214 1160.0.0.0

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[Dogway]

I know FFMS2 can't but does LSMASH support image batch loading? something like IMG_%04d.JPG?

The only image loader that I know that does this is ImageSource, but that one converts the images to RGB internally.

[kedautinh12]

You can try ImageSequence
http://avisynth.nl/index.php/ImageSequence

[Dogway]

CoronaSequence outputs RGB, I tried RawSequence() but it gives an error:
Error requesting frame 0
Cache: Filter returned invalid response to
CACHE_GETCHILD_AUDIO_MODE.10

Anyway, for today I solved it using batch scripts, but would be good to have a nice image sequence loader for next times.

[kedautinh12]

You can try JpegSource too
http://avisynth.nl/index.php/JpegSource

[Dogway]

JpegSource can load image sequences? Anyway it's an outdated (x86) closed source plugin, it's the best jpeg decoder quality wise but no help there.

[qyot27]

I know FFMS2 can't but does LSMASH support image batch loading? something like IMG_%04d.JPG?

The only image loader that I know that does this is ImageSource, but that one converts the images to RGB internally.
To answer this original question, yes it does (at least on Linux, but as filename handling should be internal to the plugin, I would assume that's also true on Windows).

I output a sequence of PNG files as well as a sequence of JPGs, and LwLibavVideoSource was totally cool opening the entire sequence using:
LwLibavVideoSource("output_%02d.jpg") # or output_%02d.png
The PNG sequence remained bgr24, the JPG sequence remained yuv420p.

[StainlessS]

at least on Linux

On W10 [EDIT: x64],

create_RGB.avs

# Export "Image Sequence" to ".\PIC_RGB" existing directory via VDUB2, png with basename="RGBIMAGE_" and 6 digits
Colorbars(Pixel_Type="RGB24")
ShowFrameNumber
Trim(0,-100)


RgbIn.avs

LwLibavVideoSource(".\PIC_RGB\RGBIMAGE_%06d.png") # "PIC_RGB" directory within the current directory.


Works OK,

###########

create_YUV.avs

# Export "Image Sequence" to ".\PIC_YUV" existing directory via VDUB2, jpeg with basename="YUVIMAGE_" and 6 digits

Colorbars(Pixel_Type="YV12")
ShowFrameNumber
Trim(0,-100)


YuvIn.avs

LwLibavVideoSource(".\PIC_YUV\YUVIMAGE_%06d.jpg") # "PIC_YUV" directory within the current directory.


Avisynth open failure.
[Fatal]: Failed to avformat_open_input.


EDIT: Oops, WORKS (returned as YV24, pre save source was YV12 [VDUB may have changed for save])
YuvIn.avs

LwLibavVideoSource(".\PIC_YUV\YUVIMAGE_%06d.jpeg") # VDub2 exported with 'jpeg' extension

[Reel.Deel]

JpegSource can load image sequences?

It cannot. SEt had it on his todo list (https://forum.doom9.org/showthread.php?p=1709220#post1709220) but never happened.

LwLibavVideoSource was totally cool opening the entire sequence using:
LwLibavVideoSource("output_%02d.jpg") # or output_%02d.png
The PNG sequence remained bgr24, the JPG sequence remained yuv420p.

Nice tip, I had no idea LwLibavVideoSource was able to do that.

-------------

Another method of opening up image sequences is ImageSourceNV (https://forum.doom9.org/showthread.php?t=184825) but requires an NVidia GPU and it converts YUV images to 4:4:4.

[flossy_cake]

I am confused about drc_scale (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/blob/master/AviSynth/README.md):



0.0 : DRC disabled. Produces full range audio.
# ok I understand this

0.0 < drc_scale <= 1.0 : DRC enabled. Applies a fraction of the stream DRC value. Audio reproduction is between full range and full compression.
# ok so 0.5 gives half compression, 1.0 gives full compression, understood

> 1.0 : DRC enabled. Applies drc_scale asymmetrically. Loud sounds are fully compressed. Soft sounds are enhanced.
# I thought 1.0 was already fully compressed? What is meant by "asymmetrically"?



:thanks:

[tebasuna51]

You can read this thread about DRC (https://forum.doom9.org/showthread.php?t=56020) and see the attached image with the volume amplify/ attenuation values (the most common is Film Standard) applied to the audio.

With drc=0 the 'None' curve is applied, with drc=1 the showed curve is applied and with drc>1 a more agresive (plane) curve must be applied with all sounds at same volume.

Of course apply drc is only recommended at play time, never when you want recode or recover the original audio.

[flossy_cake]

You can read this thread about DRC (https://forum.doom9.org/showthread.php?t=56020) and see the attached image with the volume amplify/ attenuation values (the most common is Film Standard) applied to the audio.

With drc=0 the 'None' curve is applied, with drc=1 the showed curve is applied and with drc>1 a more agresive (plane) curve must be applied with all sounds at same volume.

Of course apply drc is only recommended at play time, never when you want recode or recover the original audio.

Very useful, thanks.

I can now try to emulate these in soxfilter as well - great!

:thanks:

[tebasuna51]

I can now try to emulate these in soxfilter as well - great!

Maybe like this? (https://forum.doom9.org/showthread.php?p=779165#post779165)

[kedautinh12]

L-SMASH-Works 1161.0.0.0 (info in this link, just click. If you confused, don't use it)
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[FranceBB]

Thanks for the Christmas present, asd-g! :)

[flossy_cake]

Maybe like this? (https://forum.doom9.org/showthread.php?p=779165#post779165)

Hell yes that's just what I wanted :thanks:

[Dogway]

On W10 [EDIT: x64],

create_YUV.avs

# Export "Image Sequence" to ".\PIC_YUV" existing directory via VDUB2, jpeg with basename="YUVIMAGE_" and 6 digits

Colorbars(Pixel_Type="YV12")
ShowFrameNumber
Trim(0,-100)


YuvIn.avs

LwLibavVideoSource(".\PIC_YUV\YUVIMAGE_%06d.jpg") # "PIC_YUV" directory within the current directory.


Avisynth open failure.
[Fatal]: Failed to avformat_open_input.


EDIT: Oops, WORKS (returned as YV24, pre save source was YV12 [VDUB may have changed for save])
YuvIn.avs

LwLibavVideoSource(".\PIC_YUV\YUVIMAGE_%06d.jpeg") # VDub2 exported with 'jpeg' extension


That's the error I get for an image sequence of P1000054.JPG, P1000070.JPG, P1000074.JPG, etc.
Explicitly loading by the filename opens it but the sprint syntax calls that error. Tried with "P1000%03d.JPG", on Win11 x64.

[poisondeathray]

That's the error I get for an image sequence of P1000054.JPG, P1000070.JPG, P1000074.JPG, etc.
Explicitly loading by the filename opens it but the sprint syntax calls that error. Tried with "P1000%03d.JPG", on Win11 x64.

Is that the right numbering ?

If "gap" / "gapless" isn't supported maybe you can request that feature

What should it return ? Black "placeholder" frames inbetween ? Or just the frames present ?

[StainlessS]

Tried with "P1000%03d.JPG", on Win11 x64.
Try "P1%06d.JPG" (or for Vdub2 output "P1%06d.JPEG")
EDIT: Maybe the literal "000" of "P1000%03d.JPG" is confusing the code, somehow.

EDIT: Also if prob try,
".\P1%06d.JPG" or ".\P1%06d.JPEG",
or full path in place of current directory relative ".\".

EDIT: OK, maybe I misunderstood, looks like PDR got it.

EDIT: LWLibavVideoSource seems not to have equivalent to ImageSource("...",end=100)
LWLibavVideoSource(string source, int stream_index = -1, int threads = 0, bool cache = true, string cachefile = source + ".lwi",
int seek_mode = 0, int seek_threshold = 10, bool dr = false, int fpsnum = 0, int fpsden = 1,
bool repeat = unspecified, int dominance = 0, string format = "", string decoder = "", int prefer_hw = 0,
int ff_loglevel = 0, string cachedir = "", string ff_options = "")

[Dogway]

Thanks StainlessS, tried P1%06d.JPG, no dice.

Is that the right numbering ?

If "gap" / "gapless" isn't supported maybe you can request that feature

What should it return ? Black "placeholder" frames inbetween ? Or just the frames present ?

Frames present. These are the photos I selected from the set, and want to bulk filter them (resize, etc) with AVS+ tools.

I just tested renaming them to P1000000.JPG and P1000001.JPG and they load (both) fine. Also starting from P1000001, and so on. So the only problem it has is the numbering gaps. I can rename them to sequential temporally but could great to have this feature embedded.

[StainlessS]

Doggy, loads all files matching, "P1??????.JPG|JPEG" # "P1" + 6 characters (hopefully Digits) + "." + ("JPG" OR "JPEG")

Untested.

IN_FILES = "P1??????.JPG|JPEG" # "P1" + 6 characters (hopefully Digits) + "." + ("JPG" OR "JPEG")
OUT_FILE = "FILE.List"
result = RT_WriteFileList(IN_FILES,OUT_FILE) # Create a listing file of Pic filenames
Assert((result!=0), "No files found")
FILELIST = RT_ReadTxtFromFile(OUT_FILE) # Get list of files
#RT_FileDelete(OUT_FILE) # we are done with the temp file
NFILES = RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
RT_Debug("Found files = " + String(NFILES))

C=0 # Dummy clip
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename
tmpC = ImageSource(FN,end=0)
# tmpC = LwLibavVideoSource(FN)
C = (C.IsInt) ? TmpC : C + TmpC # All clips must be same size.
}
C # EDIT: OOps, forgot this


EDITED: still untested.

[StainlessS]

Doggy, Further to above post,


########
# RT_FSelOpen() by StainlessS
#
# Function RT_FSelOpen(string "title"="Open",string "dir"="",string "filt",string "fn="",bool "multi"=false,bool "debug"=false)
#
# Function to select EXISTING filename using GUI FileSelector.
#
# Title = Title bar text.
# Dir = Directory, "" = Current
# Filt = Lots, eg "All Files (*.*)|*.*"
# [Displayed text | wildcard] [| more pairs of Displayed text and wildcard, in pairs ONLY].
# first one is default.
# fn = Initially presented filename (if any).
# multi = Multiply Select filenames. Allows selection of more than one filename.
# debug = Send error info to DebugView window.
#
# Returns
# int, 0, user CANCELLED.
# int, non zero is error (error sent to DebugView window).
# String, Filename selected, Chr(10) separated multiline string if MULTI==true (and multiple files selected).
#
# Example, to prompt for an AVI file and play it.
# avi=RT_FSelOpen("I MUST have an AVI",filt="Avi files|*.avi")
# Assert(avi.IsString,"RT_FSelOpen: Error="+String(avi))
# AviSource(avi)
#
########

# Images, Have to be same size

TITLE = "Tell me what you want, what you really really want"
DIR = ".\"
FILT = "Image files (*.bmp;*.jpg;*.jpe;*.jpeg;*.png;*.tga;*.tif;*.gif;*.tiff)|*.bmp;*.jpg;*.jpe;*.jpeg;*.png;*.tga;*.tif;*.gif;*.tiff"
FN = ""
MULTI = True
FILELIST = RT_FSelOpen(title=TITLE,dir=DIR,Filt=FILT,fn=FN,multi=MULTI)
Assert(!(FILELIST.IsInt && FILELIST==0),"User Cancelled")
Assert(FILELIST.IsString,"RT_FSelOpen: Error="+String(FILELIST))
NFILES = RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
RT_Debug("Found files = " + String(NFILES))

C=0 # Dummy clip
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename
RT_DebugF("%d] %s",i,FN,name="ShowFilename: ")
tmpC = ImageSource(FN,end=0)
# tmpC = LwLibavVideoSource(FN)
C = (C.IsInt) ? TmpC : C + TmpC # All images must be same size.
}
C
RT_DebugF("DONE",name="FINISHED: ")
Return last


Allows selection of [EDIT: multiple] image files using GUI file selector. [must be all same size]

[Dogway]

Thanks StainlessS. The first example did the job.

I made another for loop to write images out, I hope I got it right. I don't know if the same is possible with your last example as you are not writing the filenames to any file.


EDIT: My previous code was wrong, this is my current one with ImageWriter

DIR = "D:\Fotos\Fotos 28\"
IN_FILES = DIR+"P1??????.JPG|JPEG" # "P1" + 6 characters (hopefully Digits) + "." + ("JPG" OR "JPEG")
OUT_FILE = DIR+"FILE.List"
result = RT_WriteFileList(IN_FILES,OUT_FILE) # Create a listing file of Pic filenames
Assert((result!=0), "No files found")
FILELIST = RT_ReadTxtFromFile(OUT_FILE) # Get list of files
RT_FileDelete(OUT_FILE) # we are done with the temp file
NFILES = RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
RT_Debug("Found files = " + String(NFILES))


C=0 # Dummy clip
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename
tmpC = LwLibavVideoSource(FN)

# Filtering
UV=deep_resize(tmpC,0.5,0.25,edge="Zopti",show=false) # 422 to 420
U=ExtractU(UV)
V=ExtractV(UV)
RES=deep_resize(tmpC,0.25,edge="SSIM2",show=false)

RES=CombinePlanes(ExtractY(RES),U,V, planes="YUV", pixel_type="YUV444")

RES=z_ConvertFormat(RES,pixel_type="RGBP16",colorspace_op ="470BG:auto:470BG:f=>rgb:same:same:f",resample_filter="spline16")

STRL=StrLen(FN)-4
STR=LeftStr(FN,STRL)+".PNG"
RES=ImageWriter(STR,FN,type="PNG")
C = (C.IsInt) ? RES : C + RES # All clips must be same size
RT_FileDelete(FN+".lwi") # we are done with the index file
}
C

[StainlessS]

as you are not writing the filenames to any file
Can use this, same as your current script (with separate loop or same loop).

#For(i=0,NFILES-1) { # If separate loop
FN=RT_TxtGetLine(FILELIST,i)


Your

STRL=StrLen(FN)-4
STR=LeftStr(FN,STRL)+".PNG"

only works with "JPG" not "JPEG", use

STR = RT_FilenameSplit(FN,get=7) # 7 = Drive + Dir + Name


RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the
parts joined together.
'Get' (default 15, 1 -> 15), If set, Bit 0=DRIVE, 1=Dir, 2=Name, 4=Extension.
Add 1 for Drive (bit 0), add 2 for Dir (bit 1), add 4 for Name (bit 2), add 8 for Extension (bit 3).
Some combinations do not make sense, eg Drive + Extension (1+8=9). Below sensible options.
1 = Drive (includes trailing ':')
2 = Dir (includes trailing '\')
3 = Drive + Dir
4 = Name
6 = Dir + Name
7 = Drive + Dir + Name
8 = Extension (includes leading '.')
12 = Name + Extension
14 = Dir + Name + Extension
15 = Drive + Dir + Name + Extension
Assuming a current working directory of eg "D:\avs\avi\", 'filename'="test.avi" and 'get'=15, returns "D:\avs\avi\test.avi",
so given a relative filename and default 'get'=15, is equivalent to RT_GetFullPathName(filename).


EDIT: FILELIST is a multiline string of file names in both of my examples.

[FranceBB]

Hey guys, asd released a new version of LWLibav Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/) however I'm now getting a weird behavior.
It seems that when I use LWLibavVideoSource() and LWLibavAudioSource(), the index is recreated every time even when it's already in the same directory.
Can you test it too and see if that's indeed the case or if something else changed in my systems?
It seems like an odd behavior... (I'm using AVSPmod mod 2.7.7.0 FYI).

[rgr]

Hey guys, asd released a new version of LWLibav Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/) however I'm now getting a weird behavior.
It seems that when I use LWLibavVideoSource() and LWLibavAudioSource(), the index is recreated every time even when it's already in the same directory.
Can you test it too and see if that's indeed the case or if something else changed in my systems?
It seems like an odd behavior... (I'm using AVSPmod mod 2.7.7.0 FYI).

Are there national characters (other than English) in the folder/file name?

[FranceBB]

Nope, it was a pretty standard script:


video=LWLibavVideoSource("V:\fs0\clip.dir\UDX38745.mxf")
ch12=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=1)
ch34=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=2)
ch56=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=3)
ch78=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=4)
audio=MergeChannels(ch12, ch34, ch56, ch78)
AudioDub(video, audio)

AssumeTFF()
Bob()

AssRender("V:\fs0\clip.dir\UDX38745.ass")

assumeTFF()
separatefields()
selectevery(4,0,3)
weave()


and as soon as I uncommented a filter and re-run it, it started recreating the .lwi file again for a third time:

https://i.imgur.com/151Vkzc.png

it's currently 7:46PM, so it literally just started, it will grow again until it reaches 13 MB and then if I change something in the script, it will start creating it again... O_O

[kedautinh12]

Report to Asd-g

[StvG]

Nope, it was a pretty standard script:



and as soon as I uncommented a filter and re-run it, it started recreating the .lwi file again for a third time:

https://i.imgur.com/151Vkzc.png

it's currently 7:46PM, so it literally just started, it will grow again until it reaches 13 MB and then if I change something in the script, it will start creating it again... O_O

Does some old version work without recreating the index file for this script?

The safest way is to specify individual index file for every LWLibav... call.


video=LWLibavVideoSource("V:\fs0\clip.dir\UDX38745.mxf", cachefile="V:\fs0\clip.dir\UDX38745.mxfVideo.lwi")
ch12=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=1, cachefile="V:\fs0\clip.dir\UDX38745.mxfAudio1.lwi")
ch34=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=2, cachefile="V:\fs0\clip.dir\UDX38745.mxfAudio2.lwi")
ch56=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=3, cachefile="V:\fs0\clip.dir\UDX38745.mxfAudio3.lwi")
ch78=LWLibavAudioSource("V:\fs0\clip.dir\UDX38745.mxf", stream_index=4, cachefile="V:\fs0\clip.dir\UDX38745.mxfAudio4.lwi")

[gispos]

Hey guys, asd released a new version of LWLibav Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/) however I'm now getting a weird behavior.
It seems that when I use LWLibavVideoSource() and LWLibavAudioSource(), the index is recreated every time even when it's already in the same directory.
Can you test it too and see if that's indeed the case or if something else changed in my systems?
It seems like an odd behavior... (I'm using AVSPmod mod 2.7.7.0 FYI).
I have tested it with a 40 GB mkv video file. Indexing 15 minutes.
Then sharpen(0.4) on and off. With **/avsp_split**/ approx. 0.1 seconds.

video=LWLibavVideoSource(SourceFile, indexingpr=False, cache=True, prefer_hw=1)
audio=LWLibavAudioSource(SourceFile, cache=True, av_sync=False)
audioDub(video, audio)
/**avsp_split**/
sharpen(0.4)


Without /**avsp_split**/ approx. 18 seconds, so all good for me.

--------------------

I have another problem with LWLibav and AV1 video.
My graphics card can decode AV1, but LWLibav apparently only uses software decoding.

LWLibav prefer_hw=0 or prefer_hw=1 = 15 fps
DirectShowSource = 32 fps

Here I have uploaded the used video for testing.
https://drive.google.com/drive/folders/1Q4ikSHegEgHyL3MfaXfmfHMCV4pzBQzH?usp=drive_link

Can anyone confirm this?

[StvG]

I have tested it with a 40 GB mkv video file. Indexing 15 minutes.
Then sharpen(0.4) on and off. With **/avsp_split**/ approx. 0.1 seconds.

video=LWLibavVideoSource(SourceFile, indexingpr=False, cache=True, prefer_hw=1)
audio=LWLibavAudioSource(SourceFile, cache=True, av_sync=False)
audioDub(video, audio)
/**avsp_split**/
sharpen(0.4)


Without /**avsp_split**/ approx. 18 seconds, so all good for me.

This script isn't the same as the FranceBB one because his script is using 2 audio streams - once the index file indicates one audio steam and after the same index file should indicate another audio stream -> so the index file is changed.

I have another problem with LWLibav and AV1 video.
My graphics card can decode AV1, but LWLibav apparently only uses software decoding.

LWLibav prefer_hw=0 or prefer_hw=1 = 15 fps
DirectShowSource = 32 fps

Here I have uploaded the used video for testing.
https://drive.google.com/drive/folde...usp=drive_link

Can anyone confirm this?


When you're comparing the speed of LWLibav and DirectShowSource, did you compare if DirectShowSource is frame accurate when seeking?

[qyot27]

I have another problem with LWLibav and AV1 video.
My graphics card can decode AV1, but LWLibav apparently only uses software decoding.

LWLibav prefer_hw=0 or prefer_hw=1 = 15 fps
DirectShowSource = 32 fps

Here I have uploaded the used video for testing.
https://drive.google.com/drive/folders/1Q4ikSHegEgHyL3MfaXfmfHMCV4pzBQzH?usp=drive_link

Can anyone confirm this?
At least on the Wiki, prefer_hw's arguments are only listed on the LSMASHVideoSource page despite the parameter list for LwLibavVideoSource including it, but regardless, it's something that tries to be intelligent (0 is software decoding, 1 isn't even listed as an option, 2 is Intel QSV, and 3 is a ladder that tries CUVID first, then QSV, then software decoding).

But there's no need to try to be intelligent about it if you explicitly call the correct hardware variant codec with decoder.

https://i.imgur.com/TgF1yi8l.png (https://i.imgur.com/TgF1yi8.png)


And as far as I could gather, it then also depends on what graphics card you have and how it's trying to be called. I've had issues trying to get anything (including mpv opening the file directly) to play AV1 with hwdec with D3D11 on Windows, even if FFmpeg has been linked with libvpl and has D3D11 enabled, although it generally goes several months between attempts. Maybe it finally got fixed since the last time I tried (or it was finally working in mpv the last time I tried, but I forgot).

[gispos]

I think I have chosen the right option, with h264 it works with the hardware decoding with my Nvidia card.
So it is written:

+ prefer_hw (default : 0)
Whether to prefer hardware accelerated decoder to software decoder.
Have no effect if 'decoder' is specified.
- 0 : Use default software decoder.
- 1 : Use NVIDIA CUVID acceleration for supported codec, otherwise use default software decoder.
- 2 : Use Intel Quick Sync Video acceleration for supported codec, otherwise use default software decoder.
- 3 : Try hardware decoder in the order of CUVID->QSV. If none is available then use default software decoder.

+ prefer_hw (default : 0)
Same as 'prefer_hw' of LSMASHVideoSource().

With DirectShowSource it runs over 100% faster, apparently the hardware decoding works better.
I use LavFilter, doesn't that also have something in common with libav?

[StvG]

For AV1 the threads are hardcoded (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/blob/master/common/decode.c#L137-L138) to 1 in order to be frame seeking accurate (same as ffms2 (at least the version I share)).

[gispos]

...When you're comparing the speed of LWLibav and DirectShowSource, did you compare if DirectShowSource is frame accurate when seeking?
I didn't pay attention to it, I tested it in playback and with Analisy FPS (i.e. without display playback). I have to test again, but DirectShow runs smoothly without seeking problems.

Edit:
I didn't find a position error, I set several bookmarks and jumped back and forth, and in between jumped to different positions with the slider.

DirectShowSource(SourceFile, seek=true, audio=true, video=true, convertfps=false, seekzero=false, timeout=60000)

[gispos]

I've had issues trying to get anything (including mpv opening the file directly).
Sorry, but the uploaded video was converted by google drive. So it is not the original.
I have uploaded it again in a 7z archive. https://drive.google.com/drive/folders/1Q4ikSHegEgHyL3MfaXfmfHMCV4pzBQzH?usp=drive_link

It's also a bit strange, VDub2 can't play the script cleanly, it keeps stalling.
AvsPmod and PotPlayer play the script smoothly without stalling.


DirectShowSource(SourceFile, seek=true, audio=true, video=true, convertfps=false, seekzero=false, timeout=60000)
/**avsp_split**/

ConvertBits(16)
DGHDRToSDR(white=1863, black=0, gamma=0.42, hue=0.00, r=1.00, g=1.00, b=1.15, tm=0.90, roll=0.70, fulldepth=false, impl="255", mode="pq")
ConvertToYV12(matrix="Rec709")

prefetch(4)

[StvG]

I didn't pay attention to it, I tested it in playback and with Analisy FPS (i.e. without display playback). I have to test again, but DirectShow runs smoothly without seeking problems.

Edit:
I didn't find a position error, I set several bookmarks and jumped back and forth, and in between jumped to different positions with the slider.

DirectShowSource(SourceFile, seek=true, audio=true, video=true, convertfps=false, seekzero=false, timeout=60000)

I can confirm that DirectShowSource is ok for AV1 regarding the frame seeking.

It seems from some time FFmpeg made AV1 changes so more threads can be used without affecting the seeking. You can try ffms2 5.0-RC2 (https://github.com/FFMS/ffms2/releases/tag/5.0-RC2) (it's ok AV1 seeking (https://imgbb.com/vZsFK3J) (rav1... samples)).

[qyot27]

Sorry, but the uploaded video was converted by google drive. So it is not the original.
I have uploaded it again in a 7z archive. https://drive.google.com/drive/folders/1Q4ikSHegEgHyL3MfaXfmfHMCV4pzBQzH?usp=drive_link
I don't think so? The 'new' file is roughly half the length of the previous one, which could likely account for the file size difference. But the result is still the same: directly specifying the decoder as av1_qsv lets LwLibavVideoSource open the file using the hardware decoder, and the choppiness present for CPU-only decoding disappears.

Attempts to use prefer_hw were fruitless, and upon more testing, I can verify that prefer_hw modes 3 and 2 both work fine with H264, HEVC, and VP9, but fails with AV1. It *always* selects software decoding.

Issue #48 (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/48) involved testing some decoder things and introduced the LWLDECODER variable that gets set to the codec name actually used when decoding. It can be used to verify what prefer_hw is actually doing, and the results are as follows:

decoder="av1_qsv" results in LWLDECODER == "av1_qsv" and playback is smooth
prefer_hw=3 and prefer_hw=2 both result in LWLDECODER == "libdav1d" (which is software) and playback is choppy because my i5-9400 isn't up to the task of decoding 4K 10bit AV1 smoothly, even with as optimized as libdav1d is.


And after *even more* testing, the issue is centered around libdav1d. I opened an issue on the L-SMASH-Works side about this, but if the FFmpeg that it links to has libdav1d enabled, prefer_hw will not correctly detect av1_* hardware decoders.

If the FFmpeg is built without libdav1d, then prefer_hw does work and correctly selects av1_qsv. But at the same time, if libdav1d (or libaom as well, presumably) isn't enabled, then you get no software fallback for AV1, because FFmpeg does not have a native av1 decoder. It goes from 'only able to use hw decode if you manually select the av1_* hw decoder' to 'you must use prefer_hw on AV1 so it decodes at all'.

[gispos]

I don't think so? The 'new' file is roughly half the length of the previous one, which could likely account for the file size difference. But the result is still the same...
I suspected as I had seen 1280 x 512 pixels in your picture under the information. I have just seen that you have changed the size in the script.

decoder="av1_qsv" results in LWLDECODER == "av1_qsv" and playback is smooth
My CPU does not support Intel Quick sync video. And I find it a bit strange why CUVID cannot also be selected for AV1.

Many thanks for the detailed information.

Edit:
My CPU has Quick sync but probably no AV1. Does Quick Sync have to be installed?

Edit2:
It's done. I will not use the iGPU

[qyot27]

Regardless, it was addressed in the issue over on Github (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/57). You could try with that test build.

[gispos]

Regardless, it was addressed in the issue over on Github (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/57). You could try with that test build.
The test version is very good. Thank you both qyot27 and Asd-g.
4K AV1 decode with audio and prefer_hw=3, ~100 fps, DirectShowSource just half ~50 fps.

[FranceBB]

Does some old version work without recreating the index file for this script?

Yes, the version before this one was working fine.
Effectively, it was checking whether there was an lwi file in the same directory as the file and if it was written with the same version of LWLibav.
If it was written with an older version, then it would recreate the lwi index, otherwise it would use the one it found.

In this new version, however, it recreates the lwi file every single time, not just with AVSPmod mod but also with ffmpeg and any other application.

For instance, in this very same unchanged script, we have two different command lines calling ffmpeg and it's recreating the index files both times:

video=LWLibavVideoSource("V:\fs0\clip.dir\HSM00342.mxf", cache=true)
ch12=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=1, cache=true)
ch34=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=2, cache=true)
ch56=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=3, cache=true)
ch78=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=4, cache=true)
audio=MergeChannels(ch12, ch34, ch56, ch78)
AudioDub(video, audio)


https://i.imgur.com/cDAGEIs.png


Setting cache=true doesn't affect it as it keeps recreating it anyway and it's set to true by default anyway even when it's left unspecified.

I tried with cachefile but it is still recreating it:

video=LWLibavVideoSource("V:\fs0\clip.dir\HSM00342.mxf", cache=true, cachefile="V:\fs0\clip.dir\HSM00342.mxf.lwi")
ch12=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=1, cache=true, cachefile="V:\fs0\clip.dir\HSM00342.mxf.lwi")
ch34=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=2, cache=true, cachefile="V:\fs0\clip.dir\HSM00342.mxf.lwi")
ch56=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=3, cache=true, cachefile="V:\fs0\clip.dir\HSM00342.mxf.lwi")
ch78=LWLibavAudioSource("V:\fs0\clip.dir\HSM00342.mxf", stream_index=4, cache=true, cachefile="V:\fs0\clip.dir\HSM00342.mxf.lwi")
audio=MergeChannels(ch12, ch34, ch56, ch78)
AudioDub(video, audio)


Even changing the disk didn't change a thing:

video=LWLibavVideoSource("A:\Ingest\MEDIA\temp\HSM00342.mxf", cache=true, cachefile="A:\Ingest\MEDIA\temp\HSM00342.mxf.lwi")
ch12=LWLibavAudioSource("A:\Ingest\MEDIA\temp\HSM00342.mxf", stream_index=1, cache=true, cachefile="A:\Ingest\MEDIA\temp\HSM00342.mxf.lwi")
ch34=LWLibavAudioSource("A:\Ingest\MEDIA\temp\HSM00342.mxf", stream_index=2, cache=true, cachefile="A:\Ingest\MEDIA\temp\HSM00342.mxf.lwi")
ch56=LWLibavAudioSource("A:\Ingest\MEDIA\temp\HSM00342.mxf", stream_index=3, cache=true, cachefile="A:\Ingest\MEDIA\temp\HSM00342.mxf.lwi")
ch78=LWLibavAudioSource("A:\Ingest\MEDIA\temp\HSM00342.mxf", stream_index=4, cache=true, cachefile="A:\Ingest\MEDIA\temp\HSM00342.mxf.lwi")
audio=MergeChannels(ch12, ch34, ch56, ch78)
AudioDub(video, audio)

https://i.imgur.com/AyTTwCV.png

adding a simple:

tweak(sat=1.31, dither=true)

as an example, makes it recreate the .lwi

https://i.imgur.com/VmFmSsB.png

[kedautinh12]

Why you still don't create a new issue report to the home page? Asd-g reads messages from here rarely. If you want to fix faster, you need report to the home page better than here

[FranceBB]

Who says I didn't? :P https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/58

[hello_hello]

Nope, it was a pretty standard script:

and as soon as I uncommented a filter and re-run it, it started recreating the .lwi file again for a third time:

it's currently 7:46PM, so it literally just started, it will grow again until it reaches 13 MB and then if I change something in the script, it will start creating it again... O_O

Doesn't L-Smash only index the video when indexing video, but when indexing audio, it indexes both?
FFMS2 is the same, as far as I know.

This
LWLibavVideoSource(Source, cachefile=IndexFile)
should create a smaller index file than this
LWLibavAudioSource(Source, cachefile=IndexFile)
assuming the source contains audio.

So this would be slower
v = LWLibavVideoSource(Source, cachefile=IndexFile)
a = LWLibavAudioSource(Source, cachefile=IndexFile)
AudioDub(v, a)

than this
a = LWLibavAudioSource(Source, cachefile=IndexFile)
v = LWLibavVideoSource(Source, cachefile=IndexFile)
AudioDub(v, a)

I haven't messed around with a file containing multiple audio streams myself, but what happens if you open the audio streams in reverse order, or you use the index file as the source? Unless that ability was removed at some stage, although it's working fine for r1164, which I think I'm using, at least for a file with a single audio stream.

LWLibavAudioSource(Source, cachefile=IndexFile)
a = LWLibavAudioSource(IndexFile)
v = LWLibavVideoSource(IndexFile)
AudioDub(v, a)

That's why I sometimes I use L-Smash in preference to FFMS2. You can batch index a bunch of files and open the index files directly later, without them having to be in the same folder as the source, much like indexing with DGIndex and then using the d2v file as the input for MPEG2Source. It was handy when I used to use MeGUI regularly, as it can run a batch of indexing jobs, and it understands lwi files can be used as the source.

[FranceBB]

Bug acknowledged and fixed by asd-g.
Check this commit: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/8927ea9e7d3375b057d7810d4c790ff926986e22
Test build: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/files/14545956/LSMASHSource_1177.zip

As always asd-g is amazing, he's always very committed in getting things to work and replies immediately.
Kudos to him for keeping LWLibav alive and kickin.

[leoenc]

Linux newb here, trying to build this on Ubuntu 22.04 using the following commands:
git clone https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works
cd L-SMASH-Works/AviSynth/
mkdir build
meson . build/
ninja -C build
but compilation fails with errors about a missing lsmash.h file:
ninja: Entering directory `build/'
[1/21] Compiling C++ object liblsmashsource.so.p/libavsmash_source.cpp.o
FAILED: liblsmashsource.so.p/libavsmash_source.cpp.o
c++ -Iliblsmashsource.so.p -I. -I.. -I/usr/local/include/avisynth -I/home/leo/ffmpeg_build/include -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -Wall -Winvalid-pch -Wnon-virtual-dtor -std=c++14 -O3 -Wno-sign-compare -DXXH_INLINE_ALL -D_FILE_OFFSET_BITS=64 -mfpmath=sse -msse2 -fPIC -MD -MQ liblsmashsource.so.p/libavsmash_source.cpp.o -MF liblsmashsource.so.p/libavsmash_source.cpp.o.d -o liblsmashsource.so.p/libavsmash_source.cpp.o -c ../libavsmash_source.cpp
../libavsmash_source.cpp:29:10: fatal error: lsmash.h: No such file or directory

Any help appreciated.

[qyot27]

You have to build and install L-SMASH first. Which itself requires obuparse.

https://github.com/dwbuiten/obuparse
↓
https://github.com/vimeo/l-smash
↓
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works

[leoenc]

@qyot27 thanks, I've built and installed obuparse and l-smash as you instructed and now getting the following error:
[21/21] Linking target liblsmashsource.so
FAILED: liblsmashsource.so
c++ -o liblsmashsource.so liblsmashsource.so.p/audio_output.cpp.o liblsmashsource.so.p/libavsmash_source.cpp.o liblsmashsource.so.p/lsmashsource.cpp.o liblsmashsource.so.p/lwlibav_source.cpp.o liblsmashsource.so.p/video_output.cpp.o liblsmashsource.so.p/.._common_audio_output.c.o liblsmashsource.so.p/.._common_decode.c.o liblsmashsource.so.p/.._common_libavsmash.c.o liblsmashsource.so.p/.._common_libavsmash_audio.c.o liblsmashsource.so.p/.._common_libavsmash_video.c.o liblsmashsource.so.p/.._common_lwindex.c.o liblsmashsource.so.p/.._common_lwlibav_audio.c.o liblsmashsource.so.p/.._common_lwlibav_dec.c.o liblsmashsource.so.p/.._common_lwlibav_video.c.o liblsmashsource.so.p/.._common_lwsimd.c.o liblsmashsource.so.p/.._common_osdep.c.o liblsmashsource.so.p/.._common_qsv.c.o liblsmashsource.so.p/.._common_resample.c.o liblsmashsource.so.p/.._common_utils.c.o liblsmashsource.so.p/.._common_video_output.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -shared -fPIC -Wl,-rpath,/usr/local/lib -Wl,-rpath-link,/usr/local/lib -Wl,--start-group /usr/local/lib/liblsmash.a -lm /usr/local/lib/libobuparse.so /home/leo/ffmpeg_build/lib/libavcodec.a -pthread /usr/lib/gcc/x86_64-linux-gnu/11/libatomic.so /home/leo/ffmpeg_build/lib/libfdk-aac.a /home/leo/ffmpeg_build/lib/libx264.a -lpthread -ldl /home/leo/ffmpeg_build/lib/libx265.a /usr/lib/gcc/x86_64-linux-gnu/11/libstdc++.so /usr/lib/gcc/x86_64-linux-gnu/11/libgcc_s.so /usr/lib/gcc/x86_64-linux-gnu/11/libgcc.a -lrt /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libnuma.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libz.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libva.so /home/leo/ffmpeg_build/lib/libswresample.a /home/leo/ffmpeg_build/lib/libavutil.a /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libva-drm.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libva-x11.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libvdpau.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libX11.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libdrm.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libxcb.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libXau.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libXdmcp.so /home/leo/ffmpeg_build/lib/libavformat.a /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgnutls.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libgmp.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libunistring.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libhogweed.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libnettle.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libtasn1.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libidn2.so /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libp11-kit.so /home/leo/ffmpeg_build/lib/libswscale.a -Wl,--end-group
/usr/bin/ld: /usr/local/lib/liblsmash.a(box.o): warning: relocation against `isom_fiel_box_default' in read-only section `.text'
/usr/bin/ld: /home/leo/ffmpeg_build/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Any ideas?

[qyot27]

I figured that was going to be the next error. Linking static libraries (read: ffmpeg in a custom prefix, since most guides that do that build it as static) into a shared library (liblsmashsource) requires adding -Wl,-Bsymbolic to the LDFLAGS of the shared library when you configure it.

PKG_CONFIG_PATH=$HOME/ffmpeg_build/lib/pkgconfig LDFLAGS=-Wl,-Bsymbolic meson setup <build options>

[leoenc]

@qyot27 I'm probably missing something as I ran the following and got the same error:

git clone https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works
cd L-SMASH-Works/AviSynth/
mkdir build
PKG_CONFIG_PATH=$HOME/ffmpeg_build/lib/pkgconfig LDFLAGS=-Wl,-Bsymbolic meson setup . build
ninja -C build/

[qyot27]

NVIDIA users:
git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers
sudo checkinstall --pkgname=nv-codec-headers --pkgversion="0:$(git rev-list --count HEAD)-g$(git rev-parse --short HEAD)" --backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes --fstrans=no --default

Intel users:
sudo apt-get install libvpl-dev

AV1 software decoding with libdav1d:
sudo apt-get install libdav1d-dev

FFmpeg-minimal:
Use --enable-libdav1d for AV1 software decoding
Use --enable-vpl to enable QSV hardware decoders
FFmpeg autodetects the nv-codec-headers, so the nvdec decoders should be automatically enabled
git clone https://git.videolan.org/git/ffmpeg.git
mkdir -p ffmpeg/ffmpeg-build
cd ffmpeg/ffmpeg-build
../configure --prefix=$HOME/ffmpeg_build_minimal \
--enable-gpl --enable-version3 --disable-encoders \
--disable-muxers --disable-doc --disable-debug --enable-libdav1d \
--enable-libvpl --cpu=native --extra-cflags="-march=native" && \
make -j$(nproc) && \
make install

obuparse:
git clone https://github.com/dwbuiten/obuparse.git && \
cd obuparse && \
make && \
sudo make install
The install step might throw an error if the symlink to libobuparse.so already exists; that can be safely ignored.

l-smash:
git clone https://github.com/vimeo/l-smash.git
cd l-smash
./configure --extra-cflags="-march=native -fPIC" && \
make -j$(nproc) && \
sudo checkinstall --pkgname=lsmash --pkgversion="0:$(git rev-list --count HEAD)-g$(git rev-parse --short HEAD)" --backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes --fstrans=no --default

LSMASHSource:
git clone https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works.git && \
mkdir -p L-SMASH-Works/AviSynth/build && \
cd L-SMASH-Works/AviSynth/build && \
PKG_CONFIG_PATH=$HOME/ffmpeg_build_minimal/lib/pkgconfig LDFLAGS="-Wl,-Bsymbolic" meson setup .. && \
ninja && \
sudo checkinstall --pkgname=lsmashsource --pkgversion="1:$(git rev-list --count HEAD)-g$(git rev-parse --short HEAD)" --backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes --stripso=yes --addso=yes --fstrans=no --default ninja install
(Alternatively, instead of installing it to the system, just copy liblsmashsource.so to either ~/.avisynth or ~/.local/lib/avisynth)

[leoenc]

Working great now, thanks!

[LigH]

I found a WMV which I want to convert, and in my experience, L-SMASH Works does that quite well, in general; but in this case I noticed that the audio gets async at the end of the clip. I can't check easily how much it is in the beginning, as there is only music, the interview starts after about 40 minutes, and here it is clearly out of sync in the Avisynth script. But when I play the WMV in a media player (MPC-HC with LAV Filters or ffplay), it is nicely in sync.

I wonder if the file is a result of concatenated smaller clips with originally audio streams exceeding the video streams, which are still contained in the resulting file, so an Avisynth source filter decodes them linearly, but they get skipped (or missing frames frozen) when a media player syncs to timestamps ... if that is even possible with this container.

I'll try to circumvent that for my conversion (e.g. remux to WMV with ffmpeg; remux to MKV with ffmpeg or mkvtoolnix; use DSS), yet I stay curious what might cause this difference.

Or are there possibly parameters for LwLibav(Video/Audio)Source I should test?

[Music Fan]

Could this wmv have been encoded in VFR mode ? Which might cause the async.

[LigH]

MediaInfo reports VFR mode, but that is only related to the multiplexing format. I doubt that the frame actually varies. But I am not sure how to test that with certainty.

[Music Fan]

I haven't done it for years but you can try directshowsource with convertfps=false. You should see directly if the number of frames is still the same (which does not mean that the sync will be perfect but maybe better).

[LigH]

It is indeed. Just by a few hundred frames, but the result is in sync late.

[rgr]

LSMASHVideoSource("input.mp4",fpsnum=60000,fpsden=1001)

should it give me a video file with fps 59.94 and normal length? It turns my 23s file into 6h :)

LSMASHVideoSource("input.mp4").ChangeFPS(59.94) works ok.

[Selur]

@rgr: you might want your source, since that does seem to work fine here.

[tebasuna51]

WARNING: in the last (LSMASHSource.dll 2024/04/02) LSMASHAudioSource is broken (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/63) (wrong channels and decode), use always LWLibavAudioSource than seems work fine.

[tebasuna51]

v1194 seems work now with wav files.

BTW LSMASHAudioSource don't work for all raw audio, for instance raw eac3 or aac can't be read. Use only for "MP4, MOV, [ISO Base Media] and its derived file formats."

Use always LWLibavAudioSource to decode audio.

[rgr]

@rgr: you might want your source, since that does seem to work fine here.

??? I don't understand.

[rgr]

Should I have double indexing?

Creating lwi index file 100%
Creating lwi index file 12%


v=LWLibavVideoSource("test.mp4",fpsnum=50)
AudioDub(v, LWLibavAudioSource("test.mp4"))

[Selur]

you might want your source, since that does seem to work fine here
=>
you might want to share your source, since that does seem to work fine here

[FranceBB]

Yes, if you index the video and then the audio, first it does the video, then it does the audio, hence you see two lines.
If you look at the lwi file, it gets bigger as it gets updated.
Once it's done, it won't index it again. :)

By the way, if you wanna see instances in which it may index things twice by mistake (not this case) you can just stare at the lwi index. If it gets overwritten and shrinks in size before raising again then something is wrong, but if it just grows then everything is fine.
In this case I'm sure it just grows as you're indexing the audio too, so everything is fine. ;)

[Emulgator]

I am developing a snippet to automate audio decoding across a choice of audio decoders, here to control DRC (lower, deactivate/discard, invert) for decoding purposes.
Test source is a BD .m2ts with Track 1: 2.0 .ac3 + Track 2: 5.1 .dts.
Manually giving "5.1"or "0x3f" or "FL+FR+FC+LFE+SL+SR" decodes nicely, but I need to leave the channel layout default to LWLibavAudioSource, because the script can not know whats inside the source.
"Apply" needs to hand over all parameters up to the wanted one (drc_scale) in sequence. All the other parameters seem to work, so I can hand these over.
Now layout is parsed before drc_scale, so I am searching for the LWLibavAudioSource default layout parameter to hand this over too.
The 1194 readme says "", but this won't give any audio. "-1" does not and "0" does not and "0x0" does not help either. (BTW, bool indexingpr = true is missing in readme example line, but listed in explanations.)
#...we assume last() containing video already
audsrc="LWLibavAudioSource" #Asd-g versions only, here 1194
Libavaudtrk=2 #int stream_index = -1 #-1: default stream
Libavaudcache=true #bool cache = true
Libavaudcachefile=String(audfolder)+String(audfile)+String(audpid)+String(audext)+".lwi"#string cachefile = source +".lwi"
Libavaudavsync=false #bool av_sync = false
Libavaudlayout=""#"5.1"#"0x3f"#"FL+FR+FC+LFE+SL+SR" #string layout = "" | CN ? No Audio | 5.1 -> "5.1" or "FL+FR+FC+LFE+SL+SR" or "0x3f"
Libavaudrate=0 #int rate = 0
Libavauddecoder="" #string decoder = ""
Libavaudffloglevel=0 #int ff_loglevel = 0
Libavaudcachedir="" #string cachedir = ""
Libavindexingpr=true #bool indexingpr = true
Libavauddrcscale=0.0 #float drc_scale = 1.0 #0.0 deactivates DRC, -values invert the DRC processing
Libavaudffopts="" #string ff_options = ""
Exist(String(avfolder)+String(avfile0)+String(avext0)) ? Apply(vidsrc,String(avfolder)+String(avfile0)+String(avext0))\
: Assert(false, "! NO SUCH AV FILE -> " + String(avfolder)+String(avfile0)+String(avext0) + "<- NO SUCH AV FILE !")
try {AudioDub(last,Apply(audsrc,String(audfolder)+String(audfile)+String(audpid)+String(audext),Libavaudtrk,Libavaudcache,Libavaudcachefile,Libavaudavsync,Libavaudlayout,Libavaudrate,Libavauddecoder,Libavaudffloglevel,Libavaudcachedir,Libavindexingpr,Libavauddrcscale,Libavaudffopts))}
catch (err) {last}
If I comment the try/catch lines out and run only
AudioDub(last,Apply(audsrc,String(audfolder)+String(audfile)+String(audpid)+String(audext),Libavaudtrk,Libavaudcache,Libavaudcachefile,Libavaudavsync,Libavaudlayout,Libavaudrate,Libavauddecoder,Libavaudffloglevel,Libavaudcachedir,Libavindexingpr,Libavauddrcscale,Libavaudffopts))
I get: "LWLibavAudioSource: Failed to open resampler."

Asd-g, what do I have to pass as default here ? Or is it a bug ?

[rgr]

=>
you might want to share your source, since that does seem to work fine here

Yes of course. But it's private, so I can only send it to the plugin author.

[rgr]

Yes, if you index the video and then the audio, first it does the video, then it does the audio, hence you see two lines.
If you look at the lwi file, it gets bigger as it gets updated.
Once it's done, it won't index it again. :)

Thanks.
So it would be good to introduce the "LWLibavSource" command like ffms2 or BS :)

[hello_hello]

Thanks.
So it would be good to introduce the "LWLibavSource" command like ffms2 or BS :)

I've been meaning to update this function for a long time, probably years, so it'll work with the latest L-SMASH Source.
I've only given it a quick check. It should be fine, but let me know if you find any issues.

# ===============================================================================
# LibavSource2
# ===============================================================================
#
# Similar to the original LibavSource2, but audio is enabled by default.
# Both video and audio can be disabled.
# Audio-only and video-only files can be opened without first having to
# disable the audio or video accordingly to prevent an error message.
#
# ===============================================================================

function LibavSource2(string Source, int "VTrack", int "ATrack", int "Threads", \
bool "Cache", string "CacheFile", int "Seek_Mode", int "Seek_Threshold", bool "DR", \
int "FPSNum", int "FPSDen", bool "Repeat", int "Dominance", string "Format", \
string "VDecoder", string "ADecoder", int "Prefer_HW", int "FF_LogLevel", \
string "CacheDir", string "FF_Options", bool "AV_Sync", string "Layout", int "Rate") {

VTrack = default(VTrack, -1)
ATrack = default(ATrack, -1)
Threads = default(Threads, 0)
Cache = default(Cache, true)
CacheFile = default(CacheFile, Source + ".lwi")
Seek_Mode = default(Seek_Mode, 0)
Seek_Threshold = default(Seek_Threshold, 10)
DR = default(DR, false)
FPSNum = default(FPSNum, 0)
FPSDen = default(FPSDen, 1)
Repeat = default(Repeat, false)
Dominance = default(Dominance, 0)
Format = default(Format, "")
VDecoder = default(VDecoder, "")
ADecoder = default(ADecoder, "")
Prefer_HW = default(Prefer_HW, 0)
FF_LogLevel = default(FF_LogLevel, 0)
CacheDir = default(CacheDir, "")
FF_Options = default(FF_Options, "")
AV_Sync = default(AV_Sync, false)

VideoEnabled = (VTrack > -2)
AudioEnabled = (ATrack > -2)
VideoNotFound = false
AudioNotFound = false
IsVideo = false
IsAudio = false
LS2 = " LibavSource2 " + chr(10)
VideoError = ""
AudioError = ""

assert(VideoEnabled || AudioEnabled, LS2 + " Both video and audio are disabled " + chr(10))

# -------------------------------------------------------------------------------

AudioEnabled ? Eval("""

try{

Audio = LWLibavAudioSource(Source, stream_index=ATrack, \
cache=Cache, cachefile=CacheFile, av_sync=AV_Sync, layout=Layout, rate=Rate, \
decoder=ADecoder, ff_loglevel=FF_LogLevel, cachedir=CacheDir, ff_options=FF_Options)

IsAudio = true

}catch(err_msg){

AudioError = " LWLibavAudioSource " + chr(10) + " " + err_msg + " " + chr(10)
AudioNotFound = (FindStr(LCase(err_msg), "failed to get the audio track") > 0)
NotAudio = (FindStr(LCase(err_msg), "not an audio track") > 0)
assert(!NotAudio, LS2 + " The specified audio track is not an audio track " + chr(10))
assert(AudioNotFound || NotAudio, AudioError)

} """) : nop()

# -------------------------------------------------------------------------------

VideoEnabled ? Eval("""

try{

Video = LWLibavVideoSource(Source, stream_index=VTrack, threads=Threads, \
cache=Cache, cachefile=CacheFile, seek_mode=Seek_Mode, seek_threshold=Seek_Threshold, dr=DR, \
fpsnum=FPSNum, fpsden=FPSDen, repeat=Repeat, dominance=Dominance, format=Format, decoder=VDecoder, \
prefer_hw=Prefer_HW, ff_loglevel=FF_LogLevel, cachedir=CacheDir, ff_options=FF_Options)

IsVideo = true

}catch(err_msg){

VideoError = " LWLibavVideoSource " + chr(10) + " " + err_msg + " " + chr(10)
VideoNotFound = (FindStr(LCase(err_msg), "failed to get the video track") > 0)
NotVideo = (FindStr(LCase(err_msg), "not a video track") > 0)
assert(!NotVideo, LS2 + " The specified video track is not a video track " + chr(10))
assert(VideoNotFound || NotVideo, VideoError)

} """) : nop()

# -------------------------------------------------------------------------------

assert(!VideoNotFound || AudioEnabled, LS2 + " No video found & audio is disabled " + chr(10))
assert(!AudioNotFound || VideoEnabled, LS2 + " No audio found & video is disabled " + chr(10))

IsAudio = AudioEnabled && IsAudio
IsVideo = VideoEnabled && IsVideo

return IsAudio && IsVideo ? AudioDub(Video, Audio) : IsAudio ? Audio : Video }

# ===============================================================================


Edit: LSMASHSource2 version. Tested even less.

# ===============================================================================
# LSMASHSource2
# ===============================================================================
#
# Similar to the original LSMASHSource2, but audio is enabled by default.
# Both video and audio can be disabled.
# Audio-only and video-only files can be opened without first having to
# disable the audio or video accordingly to prevent an error message.
#
# ===============================================================================

function LSMASHSource2(string Source, int "VTrack", int "ATrack", int "Threads", \
int "Seek_Mode", int "Seek_Threshold", bool "DR", int "FPSNum", int "FPSDen", \
string "Format", string "VDecoder", string "ADecoder", int "Prefer_HW", \
int "FF_LogLevel", string "FF_Options", bool "Skip_Priming", string "Layout", int "Rate") {

VTrack = default(VTrack, -1)
VTrack = (VTrack == -1) ? 0 : VTrack
ATrack = default(ATrack, -1)
ATrack = (ATrack == -1) ? 0 : ATrack
Threads = default(Threads, 0)
Seek_Mode = default(Seek_Mode, 0)
Seek_Threshold = default(Seek_Threshold, 10)
DR = default(DR, false)
FPSNum = default(FPSNum, 0)
FPSDen = default(FPSDen, 1)
Format = default(Format, "")
VDecoder = default(VDecoder, "")
ADecoder = default(ADecoder, "")
Prefer_HW = default(Prefer_HW, 0)
FF_LogLevel = default(FF_LogLevel, 0)
FF_Options = default(FF_Options, "")
Skip_Priming = default(Skip_Priming, true)

VideoEnabled = (VTrack > -2)
AudioEnabled = (ATrack > -2)
VideoNotFound = false
AudioNotFound = false
IsVideo = false
IsAudio = false
LS2 = " LSMASHSource2 " + chr(10)
VideoError = ""
AudioError = ""

assert(VideoEnabled || AudioEnabled, LS2 + " Both video and audio are disabled " + chr(10))

# -------------------------------------------------------------------------------

AudioEnabled ? Eval("""

try{

Audio = LSMASHAudioSource(Source, track=ATrack, skip_priming=Skip_Priming, \
layout=Layout, rate=Rate, decoder=ADecoder, ff_loglevel=FF_LogLevel, ff_options=FF_Options)

IsAudio = true

}catch(err_msg){

AudioError = " LSMASHAudioSource " + chr(10) + " " + err_msg + " " + chr(10)
AudioNotFound = (FindStr(LCase(err_msg), "failed to find audio track") > 0)
NotAudio = (FindStr(LCase(err_msg), "not an audio track") > 0)
assert(!NotAudio, LS2 + " The specified audio track is not an audio track " + chr(10))
assert(AudioNotFound || NotAudio, AudioError)

} """) : nop()

# -------------------------------------------------------------------------------

VideoEnabled ? Eval("""

try{

Video = LSMASHVideoSource(Source, track=VTrack, threads=Threads, seek_mode=Seek_Mode, \
seek_threshold=Seek_Threshold, dr=DR, fpsnum=FPSNum, fpsden=FPSDen, format=Format, \
decoder=VDecoder, prefer_hw=Prefer_HW, ff_loglevel=FF_LogLevel, ff_options=FF_Options)

IsVideo = true

}catch(err_msg){

VideoError = " LSMASHVideoSource " + chr(10) + " " + err_msg + " " + chr(10)
VideoNotFound = (FindStr(LCase(err_msg), "failed to find video track") > 0)
NotVideo = (FindStr(LCase(err_msg), "not a video track") > 0)
assert(!NotVideo, LS2 + " The specified video track is not a video track " + chr(10))
assert(VideoNotFound || NotVideo, VideoError)

} """) : nop()

# -------------------------------------------------------------------------------

assert(!VideoNotFound || AudioEnabled, LS2 + " No video found & audio is disabled " + chr(10))
assert(!AudioNotFound || VideoEnabled, LS2 + " No audio found & video is disabled " + chr(10))

IsAudio = AudioEnabled && IsAudio
IsVideo = VideoEnabled && IsVideo

return IsAudio && IsVideo ? AudioDub(Video, Audio) : IsAudio ? Audio : Video }

# ===============================================================================

[flossy_cake]

I noticed there is an ff_options argument... is it possible to specify an audio track language with it? Something like...

LWLibavAudioSource(file, ff_options="map=0:a:m:language:eng")

[tebasuna51]

...is it possible to specify an audio track language with it?

Inside AviSynth the audio can't have a property like language.
Your ffmpeg parameter is for a container like mkv, not for an uncompresed audio file like LWLibavAudioSource output.

[flossy_cake]

Your ffmpeg parameter is for a container like mkv, not for an uncompresed audio file like LWLibavAudioSource output.

Yep so in this case the first argument "file" points to an mkv file.

[tebasuna51]

Yep so in this case the first argument "file" points to an mkv file.

No mather the input file the LWLibavAudioSource output is uncompressed audio without a language property, and specify the language is usseless to select the track inside the mkv.

[flossy_cake]

No mather the input file the LWLibavAudioSource output is uncompressed audio without a language property, and specify the language is usseless to select the track inside the mkv.

I think there is a misunderstanding here - I was trying to tell LWLibavAudioSource to select the audio track to decode based on language instead of track number (stream_index). As far as I can tell the ff_options argument allows us to set some decoder settings which could potentially include language.

The reason I want to do this is because I have a large number of files in a series where the english track number changes from file to file and it's a pain to have to check them all manually one by one to set stream_index to the right number for each one.

I'm currently reducing the workload of this task by using a custom MediaInfo view so that it prints out the english track number of all files at a glance and then manually tagging the file names with [audio_track=n] and then inside my script if(FindStr(sourcefile, "[audio_track=n]" != 0)) setting stream_index based on the result of that.

[qyot27]

It would have been easier to just point at the documentation (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/blob/master/AviSynth/README.md):
+ ff_options (default : "")
Set the decoder options in FFmpeg.
The format is `key=value` separated by " ". (e.g. "drc_scale=0 auto_convert=0")

-map is not a decoder option, so therefore no, it won't work. The examples even show this, drc_scale is an option specific to only certain decoders (like ac3, but not aac, for example), when you call up the options list with ffmpeg -h decoder=<format-name>.

For clarity, I would suggest that ff_options be renamed to ff_decoder_options to make that explicit (especially since it would be theoretically possible to have ff_demuxer_options as well, unless the two are already rolled into ff_options and the documentation just doesn't explain that).

[Music Fan]

The reason I want to do this is because I have a large number of files in a series where the english track number changes from file to file and it's a pain to have to check them all manually one by one to set stream_index to the right number for each one.
You could also remux all your files with ffmpeg keeping only the english track*, then re-encode them.
* with this :
-map 0:a:m:language:en: -c:a copy

[flossy_cake]

You could also remux all your files with ffmpeg keeping only the english track*, then re-encode them.
* with this :
-map 0:a:m:language:en: -c:a copy

Thanks and yes I know and I was gonna do it in mkvtoolnix as well because I get better media player compatibility than ffmpeg but in the end it was a lot of work and was faster to just tag the files and I was also a bit worried remuxing might affect the audio sync somehow so I just didn't want potential trouble over such a large number of files.

[rgr]

Has anyone managed to play an AVI file (from a miniDV camera) using LWLibavVideoSource? VirtualDub crashes right away, and ffmpeg doesn't work either.

name1="Greenwich, London (16.02.2007 12.53).avi"
a=LWLibavAudioSource(name1)
a=AudioDub(LWLibavVideoSource(name1), a)
a


(I had ffms2 in the script before and it worked OK.)

[LigH]

Please check a verbose MediaInfo report, especially in which style the audio track is multiplexed. DV video supports two types, one "inside" the video stream, the other in parallel, IIRC.

[rgr]

Please check a verbose MediaInfo report, especially in which style the audio track is multiplexed. DV video supports two types, one "inside" the video stream, the other in parallel, IIRC.

General
Format : AVI
Format/Info : Audio Video Interleave
Commercial name : DV
Format profile : OpenDML
Format settings : BitmapInfoHeader / WaveFormatEx
File size : 11.1 GiB
Duration : 52 min 59 s
Overall bit rate mode : Constant
Overall bit rate : 29.9 Mb/s
Frame rate : 25.000 FPS
Recorded date : 2007-02-18 15:42:25.000
TCOD : 400000
TCDO : 31797600000
VMAJ : 4
VMIN : 0
STAT : 79493 0 3.073964 1
DTIM : 29839817 1786375168

Video
ID : 0
Format : DV
Codec ID : dvsd
Codec ID/Hint : Sony
Duration : 52 min 59 s
Bit rate mode : Constant
Bit rate : 24.4 Mb/s
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Bottom Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 2.357
Time code of first frame : 00:52:05:17
Time code source : Subcode time code
Stream size : 10.7 GiB (96%)
Encoding settings : wb mode= / white balance= / fcm=auto focus

Audio
ID : 1
Format : PCM
Format settings : Little / Signed
Codec ID : 1
Duration : 52 min 59 s
Bit rate mode : Constant
Bit rate : 1 024 kb/s
Channel(s) : 2 channels
Sampling rate : 32.0 kHz
Bit depth : 16 bits
Stream size : 388 MiB (3%)
Alignment : Aligned on interleaves
Interleave, duration : 280 ms (7.00 video frames)
Interleave, preload duration : 280 ms

[LigH]

Apparently a "Type 2" DV AVI, okay.

Your script looks a bit unusual yet probably valid. Still, try to use a different syntax:

name1="Greenwich, London (16.02.2007 12.53).avi"
a=LWLibavAudioSource(name1)
v=LWLibavVideoSource(name1)
AudioDub(v, a)

[rgr]

It doesn't change anything.

The problem is here:
LWLibavAudioSource(name1)

Without sound it works OK.

[LigH]

I suspected that. Probably the audio stream is not correctly recognised. I am not at all sure but believe to remember that 32 kHz audio was an issue. Or the 2CC marker.

For a good reason I asked for a verbose MediaInfo report. I can't count how often I already explained how to achieve that (https://forum.doom9.org/showthread.php?p=2005134#post2005134).

[rgr]

I suspected that. Probably the audio stream is not correctly recognised. I am not at all sure but believe to remember that 32 kHz audio was an issue. Or the 2CC marker.

For a good reason I asked for a verbose MediaInfo report. I can't count how often I already explained how to achieve that (https://forum.doom9.org/showthread.php?p=2005134#post2005134).

Count : 356
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Inform : Count : 356 / Count of stream of this kind : 1 / Kind of stream : General / Kind of stream : General / Stream identifier : 0 / Inform : AVI (OpenDML) (DV): 12.6 GiB, 1 h 0 min / Count of video streams : 1 / Count of audio streams : 1 / Video_Format_List : DV / Video_Format_WithHint_List : DV (Sony) / Codecs Video : DV / Audio_Format_List : PCM / Audio_Format_WithHint_List : PCM / Audio codecs : PCM / Audio_Channels_Total :
[...]
/ File extension : avi / Format : AVI / Format : AVI / Format/Info : Audio Video Interleave / Format/Extensions usually used : avi / Commercial name : AVI DV / Commercial name : DV / Format profile : OpenDML / Format settings : BitmapInfoHeader / WaveFormatEx / Internet media type : video/vnd.avi / Interleaved : Yes / File size : 13579636736 / File size : 12.6 GiB / File size : 13 GiB / File size : 13 GiB / File size : 12.6 GiB / File size : 12.65 GiB / Duration : 3631160 / Duration : 1 h 0 min / Duration : 1 h 0 min 31 s 160 ms / Duration : 1 h 0 min / Duration : 01:00:31.160 / Duration : 01:00:31:04 / Duration : 01:00:31.160 (01:00:31:04) / Overall bit rate mode : CBR / Overall bit rate mode : Constant / Overall bit rate : 29918013 / Overall bit rate : 29.9 Mb/s / Frame rate : 25.000 / Frame rate : 25.000 FPS / Frame count : 90779 / Stream size : 42672256 / Stream size : 40.7 MiB (0%) / Stream size : 41 MiB / Stream size : 41 MiB / Stream size : 40.7 MiB / Stream size : 40.70 MiB / Stream size : 40.7 MiB (0%) / Proportion of this stream : 0.00314 / Recorded date : 2007-02-11 10:05:26.000 / File creation date : 2024-08-20 22:04:08.267 UTC / File creation date (local) : 2024-08-21 00:04:08.267 / File last modification date : 2024-08-20 22:03:31.185 UTC / File last modification date (local) : 2024-08-21 00:03:31.185 / TAPE : Bernard Abbey, Space Centre / TCOD : 0 / TCDO : 36311600000 / VMAJ : 4 / VMIN : 0 / STAT : 90779 0 3.315543 1 / DTIM : 29838788 647722752
Count of video streams : 1
Count of audio streams : 1
Video_Format_List : DV
Video_Format_WithHint_List : DV (Sony)
Codecs Video : DV
Audio_Format_List : PCM
Audio_Format_WithHint_List : PCM
Audio codecs : PCM
Audio_Channels_Total : 2
[...]
File extension : avi
Format : AVI
Format : AVI
Format/Info : Audio Video Interleave
Format/Extensions usually used : avi
Commercial name : AVI DV
Commercial name : DV
Format profile : OpenDML
Format settings : BitmapInfoHeader / WaveFormatEx
Internet media type : video/vnd.avi
Interleaved : Yes
File size : 13579636736
File size : 12.6 GiB
File size : 13 GiB
File size : 13 GiB
File size : 12.6 GiB
File size : 12.65 GiB
Duration : 3631160
Duration : 1 h 0 min
Duration : 1 h 0 min 31 s 160 ms
Duration : 1 h 0 min
Duration : 01:00:31.160
Duration : 01:00:31:04
Duration : 01:00:31.160 (01:00:31:04)
Overall bit rate mode : CBR
Overall bit rate mode : Constant
Overall bit rate : 29918013
Overall bit rate : 29.9 Mb/s
Frame rate : 25.000
Frame rate : 25.000 FPS
Frame count : 90779
Stream size : 42672256
Stream size : 40.7 MiB (0%)
Stream size : 41 MiB
Stream size : 41 MiB
Stream size : 40.7 MiB
Stream size : 40.70 MiB
Stream size : 40.7 MiB (0%)
Proportion of this stream : 0.00314
Recorded date : 2007-02-11 10:05:26.000
File creation date : 2024-08-20 22:04:08.267 UTC
File creation date (local) : 2024-08-21 00:04:08.267
File last modification date : 2024-08-20 22:03:31.185 UTC
File last modification date (local) : 2024-08-21 00:03:31.185
TCOD : 0
TCDO : 36311600000
VMAJ : 4
VMIN : 0
STAT : 90779 0 3.315543 1
DTIM : 29838788 647722752

[LigH]

I hoped to see a (hexa-) decimal number of the audio format ... plain integer PCM would be a 2-char code equivalent to 1 (0001).

From now on I leave this topic to an actual developer of L-SMASH Works. I guess a debugger is required now.

[rgr]

Count : 285
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
StreamOrder : 1
Inform : Count : 285 / Count of stream of this kind : 1 / Kind of stream : Audio / Kind of stream : Audio / Stream identifier : 0 / StreamOrder : 1 / Inform : Count : 285 / Count of stream of this kind : 1 / Kind of stream : Audio / Kind of stream : Audio / Stream identifier : 0 / StreamOrder : 1 / Inform : 1 024 kb/s, 32.0 kHz, 16 bits, 2 channels, PCM (Little / Signed) / ID : 1 / ID : 1 / Format : PCM / Format : PCM / Commercial name : PCM / Format settings : Little / Signed / Format settings, Endianness : Little / Format settings, Sign : Signed / Codec ID : 1 / Codec ID/Url : http://www.microsoft.com/windows/ / Duration : 3631160 / Duration : 1 h 0 min / Duration : 1 h 0 min 31 s 160 ms / Duration : 1 h 0 min / Duration : 01:00:31.160 / Duration : 01:00:31.160 / Bit rate mode : CBR / Bit rate mode : Constant / Bit rate : 1024000 / Bit rate : 1 024 kb/s / Channel(s) : 2 / Channel(s) : 2 channels / Sampling rate : 32000 / Sampling rate : 32.0 kHz / Samples count : 116197120 / Bit depth : 16 / Bit depth : 16 bits / Delay : 0 / Delay : 00:00:00.000 / Delay : 00:00:00.000 / Delay, origin : Stream / Delay, origin : Raw stream / Delay relative to video : 0 / Delay relative to video : 00:00:00.000 / Delay relative to video : 00:00:00.000 / Stream size : 464788480 / Stream size : 443 MiB (3%) / Stream size : 443 MiB / Stream size : 443 MiB / Stream size : 443 MiB / Stream size : 443.3 MiB / Stream size : 443 MiB (3%) / Proportion of this stream : 0.03423 / Alignment : Aligned / Alignment : Aligned on interleaves / Interleave, duration : 7.00 / Interleave, duration : 280 / Interleave, duration : 280 ms (7.00 video frames) / Interleave, preload duration : 280 / Interleave, preload duration : 280 ms / ID : 1 / ID : 1 / Format : PCM / Format : PCM / Commercial name : PCM / Format settings : Little / Signed / Format settings, Endianness : Little / Format settings, Sign : Signed / Codec ID : 1 / Codec ID/Url : http://www.microsoft.com/windows/ / Duration : 3631160 / Duration : 1 h 0 min / Duration : 1 h 0 min 31 s 160 ms / Duration : 1 h 0 min / Duration : 01:00:31.160 / Duration : 01:00:31.160 / Bit rate mode : CBR / Bit rate mode : Constant / Bit rate : 1024000 / Bit rate : 1 024 kb/s / Channel(s) : 2 / Channel(s) : 2 channels / Sampling rate : 32000 / Sampling rate : 32.0 kHz / Samples count : 116197120 / Bit depth : 16 / Bit depth : 16 bits / Delay : 0 / Delay : 00:00:00.000 / Delay : 00:00:00.000 / Delay, origin : Stream / Delay, origin : Raw stream / Delay relative to video : 0 / Delay relative to video : 00:00:00.000 / Delay relative to video : 00:00:00.000 / Stream size : 464788480 / Stream size : 443 MiB (3%) / Stream size : 443 MiB / Stream size : 443 MiB / Stream size : 443 MiB / Stream size : 443.3 MiB / Stream size : 443 MiB (3%) / Proportion of this stream : 0.03423 / Alignment : Aligned / Alignment : Aligned on interleaves / Interleave, duration : 7.00 / Interleave, duration : 280 / Interleave, duration : 280 ms (7.00 video frames) / Interleave, preload duration : 280 / Interleave, preload duration : 280 ms
ID : 1
ID : 1
Format : PCM
Format : PCM
Commercial name : PCM
Format settings : Little / Signed
Format settings, Endianness : Little
Format settings, Sign : Signed
Codec ID : 1
Codec ID/Url : http://www.microsoft.com/windows/
Duration : 3631160
Duration : 1 h 0 min
Duration : 1 h 0 min 31 s 160 ms
Duration : 1 h 0 min
Duration : 01:00:31.160
Duration : 01:00:31.160
Bit rate mode : CBR
Bit rate mode : Constant
Bit rate : 1024000
Bit rate : 1 024 kb/s
Channel(s) : 2
Channel(s) : 2 channels
Sampling rate : 32000
Sampling rate : 32.0 kHz
Samples count : 116197120
Bit depth : 16
Bit depth : 16 bits
Delay : 0
Delay : 00:00:00.000
Delay : 00:00:00.000
Delay, origin : Stream
Delay, origin : Raw stream
Delay relative to video : 0
Delay relative to video : 00:00:00.000
Delay relative to video : 00:00:00.000
Stream size : 464788480
Stream size : 443 MiB (3%)
Stream size : 443 MiB
Stream size : 443 MiB
Stream size : 443 MiB
Stream size : 443.3 MiB
Stream size : 443 MiB (3%)
Proportion of this stream : 0.03423
Alignment : Aligned
Alignment : Aligned on interleaves
Interleave, duration : 7.00
Interleave, duration : 280
Interleave, duration : 280 ms (7.00 video frames)
Interleave, preload duration : 280
Interleave, preload duration : 280 ms

[LigH]

OK: Codec-ID = 1, stereo, 32000 Hz, 16 bits, Little Endian a.k.a. intel order; as simple as an audio stream in an AVI file could be. What else might be an issue? The interleaving each 7 frames? Indexing PCM in general (may cause huge indexes)?

BTW, indexing PCM audio can take a long time. This is not a crash.

[FranceBB]

Hey rgr, the developer of LWLibav is asd-g, he's very friendly and approachable and he's generally very happy to fix bugs, but he rarely checks this topic. It might be worth opening an issue on GitHub as well Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues)

That being said, stereo PCM Little Endian 32000Hz 16bit in an AVI container is indeed a rather weird combination.

[LigH]

That being said, stereo PCM Little Endian 32000Hz 16bit in an AVI container is indeed a rather weird combination.

Not in my opinion. When AVI was developed first in VfW 1.x for Windows 3.x, hardly any other audio format than PCM was imaginable, most of all no VBR audio. Little-endian 16 bit is most common on intel CPUs where Windows runs on IBM compatible PCs. Only the 32 kHz sampling rate is typical to DV, yet common enough. And long interleaving distances (each 7 video frames instead of each frame) is not a reason for trouble either.

[FranceBB]

Probably, yes. It's just that I've rarely seen 32000Hz.
It seems to be standard for DV, but I've mostly seen 44100Hz and 48000Hz for every other source, which is why I found it to be a bit unusual. I think this is probably because hardware decoders resample it to 48000Hz anyway? Dunno. When I playback miniDV on a Sony player via SDI and record them using a Blackmagic Decklink card in v210 with PCM audio they're always 48000Hz 24bit so I guess the decoder is resampling it and saving it in a different bit depth too. It doesn't surprise me as it's a bit like when you make a DolbyE 44800Hz 20bit stream go through a hardware decoder and it automatically becomes 48000Hz 24bit PCM.


Anyway it was cool to see the real file, I mean what was actually on the miniDV tape in the form of an .avi file, that's all. :)

[LigH]

Sound Blaster 1.x cards had an even more weird common sampling rate of 22222 Hz.

Anyway, I don't expect any sampling rate per se to be a reason for a software crash of a container parser, splitter, or audio decoder.

[Z2697]

Help please
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/73

[foxyshadis]

Sound Blaster 1.x cards had an even more weird common sampling rate of 22222 Hz.

Anyway, I don't expect any sampling rate per se to be a reason for a software crash of a container parser, splitter, or audio decoder.

All the way up through the Pro line; 2.x even introduced 43478 kHz for attempts to get 44100. They were all rounded to integer multiples of a small fraction of the i8051 timer they used. Only SB16 finally had true standard sampling rates.

[wonkey_monkey]

Is there anyone who could give me some pointers on how to compile L-SMASH Source from scratch with Visual Studio? I don't really know what I'm supposed to download or how to join it all together...

[Jamaika]

Is there anyone who could give me some pointers on how to compile L-SMASH Source from scratch with Visual Studio? I don't really know what I'm supposed to download or how to join it all together...
Please note that lsmash doesn't work with the latest version of ffmpeg.

[qyot27]

Is there anyone who could give me some pointers on how to compile L-SMASH Source from scratch with Visual Studio? I don't really know what I'm supposed to download or how to join it all together...

Broadly speaking, there are two dependency trees: l-smash and FFmpeg.
obuparse->l-smash
[FFmpeg dependencies*]->FFmpeg

And once those are satisfied, you can build the L-SMASH-Works/AviSynth plugin. Since all of the dependent libs are C libraries, it is most probably far easier to manage by building or installing them via MSys2 with MinGW-w64 and GCC rather than attempting to build it all through Visual Studio.

For FFmpeg, its dependencies can be a bit of a maze, but for LSMASHSource (or FFMS2) you don't need most of them, and you can trim down the size of the built FFmpeg libs so that you're not linking in things you won't even be using (encoders, muxers, stuff like that). The general compression libraries (zlib, bzip2, lzma) and possibly iconv are the only ones that would be *vital*. Then you get into more nuanced choices like whether to use libdav1d to enable AV1 decoding, or - since we are talking about LSMASHSource - if you want to enable hwdec support by using nv-codec-headers (nvidia) and/or libvpl (Intel QSV).

FFmpeg can actually be built by Visual Studio, by dropping into an MSys2 shell from inside the VS Command Prompt, and then using the --toolchain=msvc option when configuring. Not all of the other dependencies in either the FFmpeg or l-smash trees are guaranteed to be able to do that, though, which is why I said it's probably just easier to install or build them through MSys2 with MinGW rather than trying to use VS to do it all. You just need to build the plugin itself with VS.

[wonkey_monkey]

Thanks! But hmm... maybe I should just ask you instead of trying to do it myself just yet :D

Do you think they might be any way to improve loading speeds? I'm working on a project with dozens of Blu-ray rips as input - doing some tests on a subset, LibavSource2 (LSMASHVideoSource/AudioSource fails on my files) takes 1m38s before VirtualDub is ready, whereas FFMpegSource2 takes 13 seconds. But L-SMASH Source seems to have much better audio seeking so it's still my preferred option.

Also forgive my stupidity but... if I set ff_loglevel to >0, where will I find said logs? :confused:

Edit: oops... I think I mistook you for an author of L-SMASH, probably because I've seen your name on the AvisynthPlus GitHub and got mixed up. If you happen to know the answer or can hazard a guess, great, but otherwise, sorry!

[qyot27]

If I had to hazard a guess regarding the difference in indexing times and loading times, perhaps it's down to the methods by which the indices are written, and how they're read back. I can't find any actual reference to FFMS2's index format apart from it being zlib-compressed and an 'fwrite of internal structures to disk', along with mentions of how it differs between indices written by the 32-bit version vs. the 64-bit version in relation to size_t. A format like that would ostensibly be way faster to write and naturally more compact, even before taking zlib into account, but whether that's the reason for the seeking taking longer (vs. that being more related to the zlib compression part than to the way the data is formatted), I don't know.

LSMASHSource, by contrast, writes uncompressed indices in what looks like either straight XML or something very close to it. So the indices are much larger and way more verbose, and would take longer to write; but when seeking, they don't have the overhead of being compressed, and could take advantage of optimized XML parsers, leading to much lower latency.

[wonkey_monkey]

That was roughly my thinking too, although I would expect ffmpeg to decompress fully on load, and L-SMASH to fully parse to internal structures on load, so I wouldn't think there'd be much difference once they're loaded. It's just (again, assuming) the loading that's slow for L-SMASH because it's parsing that large human-readable index file.

Incidentally I think I discovered the source of my ffms2 audio seeking trouble, which is what made me switch to L-SMASH in the first place. I've created an issue on Github about it, but the gist of it is that I don't think audio is being indexed for MKV files.

[wonkey_monkey]

it's probably just easier to install or build them through MSys2 with MinGW rather than trying to use VS to do it all.

I'm trying this now but can't work out how to do a static build. Any ideas? Googling tells me it should something along the lines of

./configure --pkg-config-flags="--static" --extra-libs="-lpthread -lm" --enable-static --disable-shared --disable-debug --disable-doc --disable-ffplay --disable-ffprobe

but that still gives me a dynamically-linked ffmpeg.exe (it complains about missing DLLs unless I run it from within MSYS2) and there's no sign of any .lib files.

And "./configure --help" doesn't list "--enable-static" and "--disable-shared" as switches - instead it has "disable static [no]" and "enable-shared [no]" :confused:

https://i.imgur.com/KMEakvU.png

[LigH]

Building my AudioBoost plugin (https://github.com/LigH-de/AudioBoost), I had to learn that AviSynth plugins using the AviSynth C++ API are not ABI cross-compatible with the avisynth.dll among different compilers. Both have to be compiled with the same. The official AviSynth+ installer ships with an avisynth.dll compiled with MSVC, so compatible plugins using the C++ API have to, as well. Or you need to compile your avisynth.dll with MinGW and GCC as well as all your plugins you want to use with it.

The exception is the older C API (LoadCPlugin).

[qyot27]

I'm trying this now but can't work out how to do a static build. Any ideas? Googling tells me it should something along the lines of

./configure --pkg-config-flags="--static" --extra-libs="-lpthread -lm" --enable-static --disable-shared --disable-debug --disable-doc --disable-ffplay --disable-ffprobe

but that still gives me a dynamically-linked ffmpeg.exe (it complains about missing DLLs unless I run it from within MSYS2) and there's no sign of any .lib files.

And "./configure --help" doesn't list "--enable-static" and "--disable-shared" as switches - instead it has "disable static [no]" and "enable-shared [no]" :confused:

https://i.imgur.com/KMEakvU.png

Slightly amended from the cross-compile instructions for Linux->Windows, assuming that a folder named mpv-build-deps is in your $HOME directory on MSys2:
cd ~/mpv-build-deps && \
git clone git://source.ffmpeg.org/ffmpeg.git && \
mkdir -p ffmpeg/ffmpeg-build/{i686,amd64} && \


amd64
+++++

cd ~/mpv-build-deps/ffmpeg/ffmpeg-build/amd64 && \
../../configure --prefix=$HOME/ffmpeg_build_minimal/amd64 \
--enable-gpl --enable-version3 --disable-w32threads --disable-encoders \
--disable-muxers --disable-doc --disable-debug --disable-devices \
--disable-avdevice --enable-libdav1d --extra-cflags="-march=native" \
--target-os=mingw32 --arch=x86_64 && \
make -j$(nproc) && \
make install

i686
+++++

cd ~/mpv-build-deps/ffmpeg/ffmpeg-build/i686 && \
../../configure --prefix=$HOME/ffmpeg_build_minimal/i686 \
--enable-gpl --enable-version3 --disable-w32threads --disable-encoders --disable-muxers \
--enable-libdav1d --disable-doc --disable-debug --disable-devices --disable-avdevice \
--cpu=pentium3 --extra-cflags="-m32 -mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
--extra-ldflags="-m32 -L/mingw32/lib" \
--windres="windres -F pe-i386" --target-os=mingw32 --arch=x86 && \
make -j$(nproc) && \
make install

And the static libraries are the .a files in the $HOME/ffmpeg_build_minimal/{amd64,i686}/lib directory.



Building my AudioBoost plugin (https://github.com/LigH-de/AudioBoost), I had to learn that AviSynth plugins using the AviSynth C++ API are not ABI cross-compatible with the avisynth.dll among different compilers. Both have to be compiled with the same. The official AviSynth+ installer ships with an avisynth.dll compiled with MSVC, so compatible plugins using the C++ API have to, as well. Or you need to compile your avisynth.dll with MinGW and GCC as well as all your plugins you want to use with it.

The exception is the older C API (LoadCPlugin).
The above is about compiling FFmpeg. All the *dependencies* for LSMASHSource can be built with GCC, but the plugin shouldn't be.

[wonkey_monkey]

Slightly amended from the cross-compile instructions for Linux->Windows, assuming that a folder named mpv-build-deps is in your $HOME directory on MSys2:

Thanks, that worked... but doesn't seem to have got much further than the previous builds I was doing. They were giving me .a files already, it turned out (apart from libpostproc.a, that's a new one), and don't I need .lib files if I'm trying to link to ffmpeg from a Visual Studio C++ project? :confused:

I'm using the MSYS2 UCRT64 environment, if that means anything to anyone...

I would also still like to solve the mystery of building a fully static and self-contained ffmpeg.exe. It seems I'm fatally cursed to never really understand building so it would nice to win just once...

-------------------------------------------------------------------------------------------------

I skipped ahead a bit just tried to build L-SMASH-Works to see what would happen. First error: it needs lsmash.h. So I got the vimeo/lsmash repo, try to build that, and I get:

error MSB8020: The build tools for Visual Studio 2013 - Windows XP (Platform Toolset = 'v120_xp') cannot be found. To build using the v120_xp build tools, please install Visual Studio 2013 - Windows XP build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution".

Only there's no such "Retarget solution" option in either of the places mentioned.

Is it just me this happens to? :mad:

[qyot27]

Thanks, that worked... but doesn't seem to have got much further than the previous builds I was doing. They were giving me .a files already, it turned out (apart from libpostproc.a, that's a new one), and don't I need .lib files if I'm trying to link to ffmpeg from a Visual Studio C++ project? :confused:
.lib and .a files are just archives, and because MSVC and GCC (or Clang) are more or less compatible on the topic of C-based ABIs, you can just use the .a files if you're in MSVC and trying to link to something built by GCC/MinGW. But you can't do that if the thing that GCC built was C++, because they aren't compatible with each other's C++ ABIs.

You can literally just rename the .a to .lib and it should also work.

I'm using the MSYS2 UCRT64 environment, if that means anything to anyone...
Generally, I don't know. UCRT vs. MSVCRT isn't something I've delved into all that much.

I would also still like to solve the mystery of building a fully static and self-contained ffmpeg.exe. It seems I'm fatally cursed to never really understand building so it would nice to win just once...
I mean, how much time do you have to kill? (https://github.com/qyot27/mpv/blob/extra-new/DOCS/crosscompile-mingw-tedious.txt) You don't have to have a Linux partition set up, installing Ubuntu through WSL2 works as well.

The FFmpeg part works fine; now that I think about it, I don't think I've been able to get a correctly working full-fat mpv build since they switched to meson (and the cd-paranoia binary was borked this time around too; I need to investigate what's happening).

[wonkey_monkey]

.lib and .a files are just archives

You can literally just rename the .a to .lib and it should also work.

Ah okay, cool. I mean I have no idea which of these .a files does what but maybe I'll figure that out once I get that far. I edited my previous post to add my next frustration...

I mean, how much time do you have to kill? (https://github.com/qyot27/mpv/blob/extra-new/DOCS/crosscompile-mingw-tedious.txt) You don't have to have a Linux partition set up, installing Ubuntu through WSL2 works as well.

Eh... right now I'd just settle for knowing whether I am completely misunderstanding everything.

A static build of ffmpeg.exe means it would be self-contained and won't rely on any other DLLs, right? Am I just completely deluded to think that that's what I'd get if I follow the instructions of various posts online that say "--enable-static" and "--disable-shared"?

Whenever I try to do anything like this I just end up feeling like you have to be born with this knowledge. Someone puts a bunch of source files on Github and everyone else already seems to know exactly what to do with them in any conceivable situation and there never seems to be any documentation that's of any use to an idiot like me...

[qyot27]

A static build of ffmpeg.exe means it would be self-contained and won't rely on any other DLLs, right?
That's a hard maybe. If you build the dependencies as static, then yes as far as those go. But the toolchains might require you pass additional flags to the linker or the compiler that point it at the static versions of the compiler's libraries.

Just as an example of this, when I built things for Windows on ARM using llvm-mingw, there were a handful of libraries the compiler itself was linking against FFmpeg that I then had to fix (by either copying the .dlls around or deleting the .dll.a so that the compiler wouldn't default to the shared version instead of the static one). This will ultimately require tweaking the options given when building the toolchain.

Am I just completely deluded to think that that's what I'd get if I follow the instructions of various posts online that say "--enable-static" and "--disable-shared"?
Options on enabling or disabling either static or shared are often A) determined by how the project designed their build system in terms of which one is the default and which isn't, and B) largely only applies to that library or program itself, but not necessarily to any dependencies it was linked to.

Whenever I try to do anything like this I just end up feeling like you have to be born with this knowledge. Someone puts a bunch of source files on Github and everyone else already seems to know exactly what to do with them in any conceivable situation and there never seems to be any documentation that's of any use to an idiot like me...
Part of that is that there are several build systems (autotools, CMake, meson, waf, etc.) in common use today, and these generally have some way of querying the configuration options, which in themselves can be fairly standard and applicable to other projects that use the same build system.

This is why advice that you need to run './configure ; make ; make install' is so ubiquitous - that's the way autotools (and some custom-designed, like x264's or FFmpeg's) systems work, and for many years, autotools was the most widely-used - or at least most strongly associated - build system for *nix stuff or FOSS more generally. Those commands are wrong for CMake and meson, even if the sequence itself is correct - all of them have distinct configuration, build, and install processes.

That's changed more recently, with build systems like CMake or meson gaining a lot of popularity. Generally, any of these systems have a core set of options you can set because they're built into the build system itself (stuff like setting an install directory, or some method of setting the C or C++ compiler, etc.), and then there are project-specific ones that you need to look at the output of './configure --help', or searching CMakelists.txt for things being toggled or set with 'option' or 'set' keywords, or the list of options described in the meson_options (or meson.options) file.

And while some or most projects are probably simple enough to use a generic, default set of options, the more complex or intricate a project is, the more likely it is that the project-specific options matter, and it requires some degree of trial and error to come up with the set of options that you personally want.

[Z2697]

I personally use media-autobuild_suite to build the static FFmpeg with custom options and start from there

[wonkey_monkey]

Thanks for taking the time to write all that, qyot27.

Don't ask me how (I've written it all down in case anyone ever needs to know, but it's been long and arduous), but with your instructions and a lot of stubbornness, I've somehow bumbled my way this far, after building lsmash, zlib, xxhash, and ffmpeg:

https://i.imgur.com/B5mmI7r.png

Any idea what I should do now?:confused:

[StvG]

.lib and .a files are just archives, and because MSVC and GCC (or Clang) are more or less compatible on the topic of C-based ABIs, you can just use the .a files if you're in MSVC and trying to link to something built by GCC/MinGW. But you can't do that if the thing that GCC built was C++, because they aren't compatible with each other's C++ ABIs.

You can literally just rename the .a to .lib and it should also work.

Building static ffmpeg with GCC (MinGW) and use it for MSVC-like L-SMASH-Works will not work. Building dynamic ffmpeg libs with GCC (MinGW) and use it for MSVC-like L-SMASH-Works would be ok.

@wonkey_monkey,
1. From "x64 Native Tools Command Prompt for VS..." type "path_to_msys\msys2_shecll.cmd -ucrt64 -use-full-path". You can replace "-ucrt64" with "-mingw64" if you want but prefer ucrt64 if you wouldn't use the plugin on older OS than Windows 10.
2. Go to msys64\usr\bin and rename "link.exe" to "link_.exe"
3. Use this script (https://pastebin.com/AT2TVMnN) to build ffmpeg and its dependencies. - from the ucrt64 shell type "./the_saved_script"
Change lines 5~15 if needed.
Download dav1d settings (https://pastebin.com/NJmsBRQd). Change line #7 to the actual path+file_name.
4. Rename back "link_.exe" to "link.exe" and you can close the ucrt64 shell.
5. Download obuparse (https://github.com/dwbuiten/obuparse) and l-smash (https://github.com/vimeo/l-smash) source codes.
Download VS solution files (https://files.catbox.moe/1iyvt2.7z) for them and extract them in the root of the relevant folders. Copy "obuparse.h" in the root of the l-smash folder. Build the libs with VS. Manually move "obuparse.h" and the lib in the relevant folders where FFMpeg is installed. Manually move "lsmash.h" and the lib in the relevant folders where FFMpeg is installed.
6. In the "x64 Native Tools Command Prompt for VS..." window navigate to the L-SMASH-Works source. Type "cmake -B build_folder -DCMAKE_PREFIX_PATH="path_to_folder_where_are_include/lib_directories" -Dsome_other_options(for e.g., -DENABLE_VPX=OFF)"
By default VS solution files will be created. Open it and build the plugin.

[wonkey_monkey]

I got as far as step 3 and then:

[41/41] Linking C executable minigzip64.exe
ninja: Entering directory `build_x64'
[0/1] Install the project...-- Install configuration: "Release"
-- Installing: C:/msys64/home/David/deps/x64-Release/lib/libzlib.dll.a
-- Installing: C:/msys64/home/David/deps/x64-Release/bin/libzlib.dll
-- Installing: C:/msys64/home/David/deps/x64-Release/lib/libzlibstatic.a
-- Installing: C:/msys64/home/David/deps/x64-Release/include/zconf.h
-- Installing: C:/msys64/home/David/deps/x64-Release/include/zlib.h
-- Installing: C:/msys64/home/David/deps/x64-Release/share/man/man3/zlib.3
-- Installing: C:/msys64/home/David/deps/x64-Release/share/pkgconfig/zlib.pc

mv: cannot stat 'zlibstatic.lib': No such file or directory


There are two copies of libstatic.a in the following locations:


C:\msys64\home\David\zlib-1.3.1\build_x64\libzlibstatic.a
C:\msys64\home\David\deps\x64-Release\lib\libzlibstatic.a

[StvG]

I got as far as step 3 and then:

[41/41] Linking C executable minigzip64.exe
ninja: Entering directory `build_x64'
[0/1] Install the project...-- Install configuration: "Release"
-- Installing: C:/msys64/home/David/deps/x64-Release/lib/libzlib.dll.a
-- Installing: C:/msys64/home/David/deps/x64-Release/bin/libzlib.dll
-- Installing: C:/msys64/home/David/deps/x64-Release/lib/libzlibstatic.a
-- Installing: C:/msys64/home/David/deps/x64-Release/include/zconf.h
-- Installing: C:/msys64/home/David/deps/x64-Release/include/zlib.h
-- Installing: C:/msys64/home/David/deps/x64-Release/share/man/man3/zlib.3
-- Installing: C:/msys64/home/David/deps/x64-Release/share/pkgconfig/zlib.pc

mv: cannot stat 'zlibstatic.lib': No such file or directory


There are two copies of libstatic.a in the following locations:


C:\msys64\home\David\zlib-1.3.1\build_x64\libzlibstatic.a
C:\msys64\home\David\deps\x64-Release\lib\libzlibstatic.a


Open zlib-xxx folder and go to build_xxxx, open CMakeCache.txt and check what is CMAKE_C_COMPILER. It seems you're building with GCC instead of cl/clang-cl.

Edit: Do not remove lines #5, 6, 7 if you don't use clang-cl just change "clang-cl" to "cl" and remove only line #7.
Edit1: Maybe also uncomment lines 53-55.

[wonkey_monkey]

Open zlib-xxx folder and go to build_xxxx, open CMakeCache.txt and check what is CMAKE_C_COMPILER. It seems you're building with GCC instead of cl/clang-cl.

CMAKE_C_COMPILER:FILEPATH=C:/msys64/ucrt64/bin/cc.exe

I commented out the line numbers you mentioned (plus line 41, "fi"), since I've never dealt with Clang.

Edit: Do not remove lines #5, 6, 7 if you don't use clang-cl just change "clang-cl" to "cl" and remove only line #7.

Trying this next.

[wonkey_monkey]

After installing meson (mingw-w64-ucrt-x86_64-meson), I've now got:

Could not find any valid candidate for cross files: C:/uc/dav1d_cross.txt

ERROR: Cannot find specified cross file: C:/uc/dav1d_cross.txt

[StvG]

Ops, forgot it. Here (https://pastebin.com/NJmsBRQd). Save it and change the path in the script. Rename "clang-cl" with "cl" and remove "strip" and "linker" lines.

Edit: Also lines 272, 273 must be removed or change it to "cl" instead "clang-cl"

[wonkey_monkey]

Thanks.

Hmm. This is odd. On running the script again I got:

Cloning into 'ffmpeg'...
warning: Could not find remote branch custom-patches-for-lsmashsource-1 to clone.
fatal: Remote branch custom-patches-for-lsmashsource-1 not found in upstream origin


I deleted all the created folders and tried again but it only got a few steps in - definitely not as far as last time - before repeating the above.

[StvG]

Change line #54 from "custom-patches-for-lsmashsource-1" to "custom-patches-for-lsmashsource".

[wonkey_monkey]

Ah I see, those were the lines I uncommented. Now the error makes sense.

I got another clang-cl error so I changed the PKG_CONFIG_PATH definition (lines 272 and 273).

I now see what looks like ffmpeg ./configure completing, and then:

WARNING: using libmfx without pkg-config
WARNING: libmfx is deprecated. Please run configure with --enable-libvpl to use libvpl instead.
GEN libavfilter/libavfilter.version
GEN libavformat/libavformat.version
GEN libavcodec/libavcodec.version
GEN libswresample/libswresample.version
GEN libswscale/libswscale.version
GEN libavutil/libavutil.version
CC libavfilter/allfilters.o
CC libavfilter/buffersrc.o
CC libavfilter/avfiltergraph.o
allfilters.c
CC libavfilter/audio.o
buffersrc.c
CC libavfilter/avfilter.o
avfiltergraph.c
CC libavfilter/buffersink.o
audio.c
C:/msys64/ucrt64/bin/../include\math.h(202): error C2065: '__asm__': undeclared identifier
C:/msys64/ucrt64/bin/../include\math.h(202): error C2146: syntax error: missing ';' before identifier '__volatile__'


followed by another 2000 or lines of errors, mostly about the word 'type' being a syntax error. IIRC Visual Studio doesn't do inline assembler on x64...

[StvG]

Do you have "nasm" installed?

[wonkey_monkey]

Do you have "nasm" installed?

Only in MSYS2. Do I need to install it Windows-wide and integrate it into Visual Studio or something?

[StvG]

Try with installing it on the Windows-wide and reopen the ucrt64 shell from vs cmd to get nasm in the path.

Edit: I will try it myself now with cl (I always use clang-cl) to see if there are errors.

Edit1: No issues from my side. ffmpeg is build.

Edit2: I updated the script link in the instruction post so to be more user friendly.

[wonkey_monkey]

Try with installing it on the Windows-wide and reopen the ucrt64 shell from vs cmd to get nasm in the path.

Edit: I will try it myself now with cl (I always use clang-cl) to see if there are errors.

Done (and I added the path to the system-wide nasm.exe in environment variables), but with no change to the errors. MSYS2's nasm.exe was already in MSYS2's path.

Perhaps I should get clang...

[StvG]

Mmm. MSYS2 nasm shouldn't be issue because when I type in the ucrt64 shell "where nasm" I get "C:\msys64\ucrt64\bin\nasm.exe". You need "mingw-w64-ucrt-x86_64-nasm".

If you didn't see the other edits from my previous post I updated the script to be more friendly. Also you can use vcpkg (https://github.com/microsoft/vcpkg) to build ffmpeg with MSVC much more easily.

Edit: for vcpkg you can see an example here (https://github.com/vapoursynth/bestsource?tab=readme-ov-file#windows-compilation).

[wonkey_monkey]

Yup, that's the same result I get from "where nasm". But I haven't done anything to tell Visual Studio/cl.exe to use nasm when it finds inline assembler, if that can even be done...

I tried with the new version of script, with these changes:

compiler="cl"
cmake_path="/c/Program Files/CMake/bin/cmake"
dav1d_cross_file="/c/msys64/home/David/dav1d_cross.txt"

but still get the errors.

[StvG]

Share "config.log" from "ffmpeg\ffbuild".

Also "config.h" and "config_components.h", and "config.asm" from "ffmpeg".

[wonkey_monkey]

Share "config.log" from "ffmpeg\ffbuild".

Also "config.h" and "config_components.h", and "config.asm" from "ffmpeg".

https://horman.net/avisynth/ffmpeg_stuff.zip

[StvG]

Thanks. Found the issue (I thought modifying mingw header was only necessary clang-cl). Use this script https://pastebin.com/gJTFMdzd

[wonkey_monkey]

Thanks. Found the issue (I thought modifying mingw header was only necessary clang-cl). Use this script https://pastebin.com/gJTFMdzd

This gets me further, thanks (and thank you very much for your continued efforts and patience).

I was getting a few instances of this:

Cannot open include file: 'stdalign.h'

I haven't found a definite solution so far, but it seems stdalign.h isn't included/supported in Visual Studio. The file does exist within MSYS2 though - I added the path to --extra-cflags in PKG_CONFIG_PATH, and now I get:

Compiler lacks support for C11 static assertions

Here's ffbuild\config.log: https://pastebin.com/Fk7Feuc2

[StvG]

I have it - https://i.ibb.co/VcyPrtKQ/Untitled.png. Try to install the latest Windows SDK or reinstall the one you have.

[wonkey_monkey]

I have it - https://i.ibb.co/VcyPrtKQ/Untitled.png. Try to install the latest Windows SDK or reinstall the one you have.

Yup, i installed the latest SDK and it has the file.

I'll try running the script again tomorrow to see if I still get the same C11 static error.

[StvG]

In case you didn't already you can delete from line 42 to 207 from the script in order to run only the ffmpeg part.

[wonkey_monkey]

Okay, that all seems to have worked!

So now I need to:

4. While you're still in the ucrt64 shell, download lsmash (vimeo) and obuparse and build them, install them in the same location as ffmpeg.

I don't actually know where ffmpeg has installed itself or how to install the other two things there (yes, I really am this clueless).

What are the files that L-SMASH Works will be looking for in "DCMAKE_PREFIX_PATH="path_to_folder_where_are_include/lib_directories" when I get that far? :confused:

[qyot27]

I don't actually know where ffmpeg has installed itself or how to install the other two things there (yes, I really am this clueless).
The value of --prefix is where ffmpeg gets installed when the 'make install' step is performed. If you don't know which directory that is (meaning, you didn't specify it), look inside the script to see what it was setting for that option.

What are the files that L-SMASH Works will be looking for in "DCMAKE_PREFIX_PATH="path_to_folder_where_are_include/lib_directories" when I get that far? :confused:
CMAKE_PREFIX_PATH sets a search path for an entire tree of headers, libs, and such, so it depends on what exactly the next steps actually look for. In some cases, it's the pkgconfig files in lib/pkgconfig, in others, the CMake modules in lib/cmake, in still other cases it's just a way of saying: look at this top-level directory and pull any .h and .a files from the include/ and lib/ folders.

[StvG]

I don't actually know where ffmpeg has installed itself or how to install the other two things there (yes, I really am this clueless).

The script creates folder "deps\x64-Release" next to the script. Search for it.

What are the files that L-SMASH Works will be looking for in "DCMAKE_PREFIX_PATH="path_to_folder_where_are_include/lib_directories" when I get that far? :confused:

FFmpeg include files/libraries, external libs/include files, liblsmash include file/lib, obuparse include file/lib...

[wonkey_monkey]

Okay... for obuparse, since it doesn't have a configure script, I edited the Makefile and just changed the first line to

PREFIX ?= ~/deps/x64-release

Then did make and make install.

For l-smash, I did

./configure --prefix="~/deps/x64-release" --extra-cflags=-I/home/david/deps/x64-Release/include --extra-ldflags=-L/home/david/deps/x64-Release/lib --includedir=~/deps/x64-Release/include --libdir=~/deps/x64-Release/lib

(I'm not sure why I couldn't use ~ as a prefix for the extra cflags and extra ldflags, but it didn't work so I specified the full path)

make/make install put .h and .a files in the x64-Release/include and x64-Release/lib directories.

Next:

cmake -B build_folder -DCMAKE_PREFIX_PATH="C:\msys64\home\David\deps\x64-Release" -DENABLE_VPX=OFF

This worked, although it complained about not finding Git or PkgConfig.

Trying to build the created .sln resulted in four unresolved externals:

__mingw_fprintf
__mingw_sprintf
___chkstk_ms
__mingw_vfprintf

[StvG]

Actually I build obuparse with created VS solution files and manually copy "obuparse.h" and the lib in "deps\x64-Release\relevant_dirs". Also I modified lsmash VS solution files to work correctly with vimeo version and again manually copy "lsmash.h" and "liblsmash.lib" to "deps\x64-Release\relevant_dirs". If you prefer VS solution files, I can share them so you can build them with VS.

[StvG]

__mingw_fprintf
__mingw_sprintf
___chkstk_ms
__mingw_vfprintf

This suggests that you're linking against ffmpeg/external libs created with GCC and not VS. For what libs are these errors?

Edit: Oh, for obuparse and liblsmash I guess you didn't explicitly used cl instead GCC. You can do in the shell: "export CC=cl".

[qyot27]

It's not necessary to specify --includedir= or --libdir= if you've set --prefix. Those are automatically populated by whatever the prefix is, and you only really want to change them if you deliberately want them somewhere else.

And the extra-{cflags,ldflags} arguments can understand $HOME, e.g., --extra-cflags=-I$HOME/deps/x64-Release/include.

Not finding Git or pkgconf sounds like you haven't added MSys2's /usr/bin to the %PATH% (or alternatively, launched MSys2 from within the VS Command Prompt so the MSys2 environment can see MSVC).

[wonkey_monkey]

Ah, all the errors are for liblsmash! I probably need to change the compiler to cl. Please hold...

[StvG]

Ah, all the errors are for liblsmash! I probably need to change the compiler to cl. Please hold...

obuparse has hardcoded GCC in the Makefile.

Here (https://filebin.net/0sgmh4a8z49pczi9) are the VS solution files for obuparse and l-smash. Extract them in the root of the relevant folders and build the libs from there.

Edit: Copy "obuparse.h" in the root of l-smash folder.

[StvG]

Not finding Git or pkgconf sounds like you haven't added MSys2's /usr/bin to the %PATH% (or alternatively, launched MSys2 from within the VS Command Prompt so the MSys2 environment can see MSVC).

For L-SMASH-Works - it checks if pkgconfig is available and if not it finds the libs with CMAKE_PREFIX_PATH. This is just warning that pkgconfig is not used. Part of the code (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/blob/master/CMakeLists.txt#L121-L128).

[wonkey_monkey]

qyot27, StvG, thank you both again for your indefatigability. I would have given up on me a long time ago. I have now successfully - if still somewhat mystifyingly - built LSMASHSource.dll.

Unfortunately while it does create a .lwi file, VirtualDub2 simply ends without an error when I try to open a file with it.

I can't build a Debug version; I can build RelWithDebInfo, but when I try to Start Debugging I get this:

https://i.imgur.com/oYtxTi5.png

even though I specified VirtualDub2 as the program to launch for debugging :confused:

[StvG]

You can build a Debug version but change in VS->C/C++->Code Generation->Runtime library to MD instead MDd.

Open VirtualDub2 (without loading anything), in VS->Debug->Attach to process and choose VirtualDub2, then load the script in VD2.

[wonkey_monkey]

I had no idea you could attach like that. Useful!

That tells me:

Exception thrown at 0x00007FF94535F270 (LSMASHSource.dll) in VirtualDub64.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

but also

Source information is missing from the debug information for this module

so I don't know where in the code the problem is happening.

I'll try again tomorrow.

[StvG]

I had no idea you could attach like that. Useful!

That tells me:

Exception thrown at 0x00007FF94535F270 (LSMASHSource.dll) in VirtualDub64.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

but also

Source information is missing from the debug information for this module

so I don't know where in the code the problem is happening.

I'll try again tomorrow.

If you build RelWithDebInfo version you probably better to disable the optimizations from O2 to Od so it will be almost the same as Debug version.
Put some break point in the lsmashworks code and see if it's hit - if it's hit with f10 go from there and see where the crash is, if it's not hit put the break point somewhere before in order to be hit.
Btw you can share a sample of the problematic video.

[wonkey_monkey]

Here, apparently, is the problem:

https://i.imgur.com/PuTNmAc.png

To clarify, the videos I'm trying to open all work fine with the distributed LSMASHSource.dll; it's just the one I've built that isn't working.

[StvG]

Are you trying to open HEVC video?

Also go to "msys64\ucrt64\include\_mingw.h" - find the line "# define _CRTIMP __declspec(dllimport)" and uncomment it if its not.

[StvG]

Here (https://filebin.net/dnqags3uipsbm44k) you can download L-SMASH-Works 1194 source code with included external libs ("deps_lsmash" folder). Just open "build" folder and the solution file.

Your build can crash because of the some later commits after the latest release. Also the ffmpeg version is different - the latest version (https://github.com/HomeOfAviSynthPlusEvolution/FFmpeg/tree/custom-patches-for-lsmashsource), older version for 1194 (https://github.com/HomeOfAviSynthPlusEvolution/FFmpeg/tree/77b972bfedef6de7b40939b0a498fa851c45ee88).

[wonkey_monkey]

Thanks for the download - but I'm afraid I'm still too dim :confused:

I tried opening LSMASHSourceVCX.sln, added the deps_lsmash\x64-Release\include and deps_lsmash\x64-Release\lib directories but got 107 unresolved externals, all seemingly MinGW related.

I also tried just opening the main folder in VS, but CMake generation fails:

1> [CMake] Could NOT find FFMPEG (missing: FFMPEG_INCLUDE_DIRS FFMPEG_LIBRARIES
1> [CMake] avcodec avformat swscale swresample avutil)

Please feel free to give up - I had to remux all of my video files anyway to remove superfluous tracks to save space, which gave me the opportunity to change them from .mkv to .mp4, which works around the issue I was having with FFMpegSource in the first place, so I can use that instead for now.

[StvG]

You don't open the solution file from folder build - https://i.ibb.co/cXND9wFG/Untitled.png
You don't need to add the path of the external libs. They are already configured for the correct paths.

[wonkey_monkey]

Like I said... dim... :o

It now works, thank you!

[FranceBB]

asd-g has been working on a new feature and released a new test build here: Link (https://github.com/user-attachments/files/18684293/LSMASHSource.zip)
His commit is here: Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/7dc8ba9295aafa5e99a59903cdd62c4fffc512a5)
The new build he made introduced a new parameter in LWLibavAudioSource(), namely "fill_audio_gaps", which is a boolean and it can be set to either "true" or "false" (by default is "false" for backwards compatibility with the current behavior).
What it does when it's set to true is basically read the PTS while it's indexing and - if the audio track has a missing sample due to some issues - it fills the chunk with silence so that the audio stays synced to the video and ends up having the same length/duration instead of being shorter and ending up desynced from the point of the first issue onward.

I tested it and it worked like a charm.

The source was:

Video: FULL HD 1920x1080 MPEG-2 High 4:2:2 Profile, Level High 50 Mbit/s yv16 25i TFF BT709 SDR
Audio: 4 Stereo 2ch PCM 24bit 48000Hz tracks (i.e 8 channels)
Container: mxf

where the video had 1500 frames (i.e 1 minute) and the audio had 1499 frames (i.e 59 seconds and 960ms).
In other words, using a 25fps based timecode format, we have:

Video: 00:00:00:00 - 00:00:59:24
Audio: 00:00:00:00 - 00:00:59:23

video=LWLibavVideoSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf")
ch12=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=1, fill_audio_gaps=true)
ch34=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=2, fill_audio_gaps=true)
ch56=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=3, fill_audio_gaps=true)
ch78=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=4, fill_audio_gaps=true)
audio=MergeChannels(ch12, ch34, ch56, ch78)
AudioDub(video, audio)

worked like a charm and filled the gap:

Video: 00:00:00:00 - 00:00:59:24
Audio: 00:00:00:00 - 00:00:59:24


The command line I tested it with was:

ffmpeg.exe -hide_banner -i "HNC06683_Audio_1_frame_shorter_than_video.mxf.avs" -pix_fmt yuv422p -vcodec mpeg2video -s 1920:1080 -aspect 16:9 -vf setfield=tff -flags +ildct+ilme+cgop -b_strategy 0 -mpv_flags +strict_gop -sc_threshold 1000000000 -r 25 -b:v 50000k -minrate 50000k -maxrate 50000k -bufsize 36408333 -qmin 5 -qmax 28 -qsquish 99 -g 12 -bf 2 -profile:v 0 -level:v 2 -color_range 1 -color_primaries 1 -color_trc 1 -colorspace 1 -an -f mxf -y "raw_video.mxf"

ffmpeg.exe -hide_banner -i "HNC06683_Audio_1_frame_shorter_than_video.mxf.avs" -c:a pcm_s24le -ar 48000 -f wav -y "audio.wav"

ffmpeg.exe -hide_banner -i "raw_video.mxf" -i "audio.wav" -c:v copy -c:a copy -f mxf -y "pre-final_output.mxf"

bmxtranswrap.exe -p -y 10:00:00:00 -t op1a --track-map stereo -o "final_output.mxf" "pre-final_output.mxf"

pause


The Mediainfo of the output file with fill_audio_gaps=true was perfect:


General
Complete name : A:\MEDIA\temp\final_output_with_padding.mxf
Format : MXF
Commercial name : XDCAM HD422
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 423 MiB
Duration : 1 min 0 s
Overall bit rate : 59.1 Mb/s
Frame rate : 25.000 FPS
Encoded date : 2025-02-12 08:41:08.112
Writing application : BBC bmx 1.3.1.0.2
Writing library : libMXF (Win64) 1.3.1.0.2
Video
ID : 1001
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings : BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Default
Format settings, GOP : M=3, N=12
Format settings, picture structure : Frame
Format settings, wrapping mode : Frame
Codec ID : 0D01030102046001-0401020201040300
Duration : 1 min 0 s
Bit rate mode : Constant
Bit rate : 50.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Time code of first frame : 00:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Closed
Stream size : 358 MiB (85%)
Title : V1
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Audio #1
ID : 2001
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 1 min 0 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A1
Locked : Yes
Audio #2
ID : 2002
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 1 min 0 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A2
Locked : Yes
Audio #3
ID : 2003
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 1 min 0 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A3
Locked : Yes
Audio #4
ID : 2004
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 1 min 0 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A4
Locked : Yes
Other #1
ID : 901-Material
Type : Time code
Format : MXF TC
Frame rate : 25.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:59:24
Time code settings : Material Package
Time code, stripped : Yes
Title : TC1
Other #2
ID : 901-Source
Type : Time code
Format : MXF TC
Frame rate : 25.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:59:24
Time code settings : Source Package
Time code, stripped : Yes
Title : TC1



While using fill_audio_gaps=false to get the old behavior creates the wrong output, as expected:

video=LWLibavVideoSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf")
ch12=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=1, fill_audio_gaps=false)
ch34=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=2, fill_audio_gaps=false)
ch56=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=3, fill_audio_gaps=false)
ch78=LWLibavAudioSource("A:\MEDIA\temp\HNC06683_Audio_1_frame_shorter_than_video.mxf", stream_index=4, fill_audio_gaps=false)
audio=MergeChannels(ch12, ch34, ch56, ch78)
AudioDub(video, audio)

In the mediainfo report we can see that the video is truncated by 1 frame due to the unpadded audio:

General
Complete name : A:\MEDIA\temp\final_output_without_padding.mxf
Format : MXF
Commercial name : XDCAM HD422
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 423 MiB
Duration : 59 s 960 ms
Overall bit rate : 59.2 Mb/s
Frame rate : 25.000 FPS
Encoded date : 2025-02-12 08:23:13.332
Writing application : BBC bmx 1.3.1.0.2
Writing library : libMXF (Win64) 1.3.1.0.2
Video
ID : 1001
Format : MPEG Video
Commercial name : XDCAM HD422
Format version : Version 2
Format profile : 4:2:2@High
Format settings : CustomMatrix / BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format settings, picture structure : Frame
Format settings, wrapping mode : Frame
Codec ID : 0D01030102046001-0401020201040300
Duration : 59 s 960 ms
Bit rate mode : Constant
Bit rate : 50.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.965
Time code of first frame : 01:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Closed
Stream size : 357 MiB (84%)
Title : V1
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Audio #1
ID : 2001
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 59 s 960 ms
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A1
Locked : Yes
Audio #2
ID : 2002
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 59 s 960 ms
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A2
Locked : Yes
Audio #3
ID : 2003
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 59 s 960 ms
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A3
Locked : Yes
Audio #4
ID : 2004
Format : PCM
Format settings : Little
Format settings, wrapping mode : Frame (BWF)
Codec ID : 0D01030102060100
Duration : 59 s 960 ms
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 25.000 FPS (1920 SPF)
Bit depth : 24 bits
Stream size : 16.5 MiB (4%)
Title : A4
Locked : Yes
Other #1
ID : 901-Material
Type : Time code
Format : MXF TC
Frame rate : 25.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:59:23
Time code settings : Material Package
Time code, stripped : Yes
Title : TC1
Other #2
ID : 901-Source
Type : Time code
Format : MXF TC
Frame rate : 25.000 FPS
Time code of first frame : 10:00:00:00
Time code of last frame : 10:00:59:23
Time code settings : Source Package
Time code, stripped : Yes
Title : TC1




Up until now I've been indexing all files with fill_audio_gaps=true and I haven't seen any issues with leaving it on at all times. I mean, if there aren't any gaps to fill it just doesn't do anything and if there are then it does what it's supposed to do. Although the chances of having issues with audio are slim in most containers, this is extremely useful for people who deal with files in .ts and .mxf containers all the time as it's very easy to have CRC Errors in live recordings, especially when the signal isn't great.

[filler56789]

asd-g has been working on a new feature and released a new test build here: Link (https://github.com/user-attachments/files/18684293/LSMASHSource.zip)
His commit is here: Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/7dc8ba9295aafa5e99a59903cdd62c4fffc512a5)
The new build he made introduced a new parameter in LWLibavAudioSource(), namely "fill_audio_gaps", which is a boolean and it can be set to either "true" or "false" (by default is "false" for backwards compatibility with the current behavior).
What it does when it's set to true is basically read the PTS while it's indexing and - if the audio track has a missing sample due to some issues - it fills the chunk with silence so that the audio stays synced to the video and ends up having the same length/duration instead of being shorter and ending up desynced from the point of the first issue onward.

That's an overdue and very-welcome update. :goodpost:
It's very similar to what Lord Mulder's wma2wav.exe does to WMA streams when converting them to uncompressed .WAV files.

[rgr]

asd-g has been working on a new feature and released a new test build here: Link (https://github.com/user-attachments/files/18684293/LSMASHSource.zip)
His commit is here: Link (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/7dc8ba9295aafa5e99a59903cdd62c4fffc512a5)
The new build he made introduced a new parameter in LWLibavAudioSource(), namely "fill_audio_gaps", which is a boolean and it can be set to either "true" or "false" (by default is "false" for backwards compatibility with the current behavior).
What it does when it's set to true is basically read the PTS while it's indexing and - if the audio track has a missing sample due to some issues - it fills the chunk with silence so that the audio stays synced to the video and ends up having the same length/duration instead of being shorter and ending up desynced from the point of the first issue onward.

I tested it and it worked like a charm.

I also tested it and it didn't work properly with my mp4 file with audio gaps from recorder. In fact, even worse than with fill_audio_gaps set to false. With false, audible desync occurred after 4 hours. With true, it's bad after 1 hour. Only EAC3TO still handles it and fills gaps correctly.
I also checked on m2ts and avi files (from camera) -- also without success. But only ffmpeg can handle these files, no AviSynth plugin works properly.

[Jamaika]

I don't know how to compile ffmpeg 7.2.0-1d2c391.
Are add-ons like the latest avisynth plus plugins or lcevc decoder definitively deleted in the ffmpeg 7.2.0 2025?

base_decoder_libav.cpp: In member function 'void lcevc_dec::utility::BaseDecoderLibAV::close()':
base_decoder_libav.cpp:361:9: error: 'avcodec_close' was not declared in this scope; did you mean 'avio_close'?
361 | avcodec_close(m_videoDecCtx);
| ^~~~~~~~~~~~~
| avio_close

In file included from convert.cpp:103:
plugins/video_output.cpp: In function 'void avs_set_frame_properties(AVFrame*, AVStream*, int64_t, int64_t, bool, PVideoFrame&, int, int, IScriptEnvironment*, int)':
plugins/video_output.cpp:890:20: error: 'AVFrame' {aka 'struct AVFrame'} has no member named 'interlaced_frame'
890 | if ( av_frame->interlaced_frame )
| ^~~~~~~~~~~~~~~~
plugins/video_output.cpp:891:33: error: 'AVFrame' {aka 'struct AVFrame'} has no member named 'top_field_first'
891 | field_based = av_frame->top_field_first ? 2 : 1;
| ^~~~~~~~~~~~~~~
plugins/video_output.cpp:925:37: error: 'AVStream' {aka 'struct AVStream'} has no member named 'nb_side_data'
925 | for (int i = 0; i < stream->nb_side_data; ++i)
| ^~~~~~~~~~~~
plugins/video_output.cpp:927:25: error: 'AVStream' {aka 'struct AVStream'} has no member named 'side_data'
927 | if (stream->side_data[i].type == AV_PKT_DATA_MASTERING_DISPLAY_METADATA)
| ^~~~~~~~~
plugins/video_output.cpp:929:114: error: 'AVStream' {aka 'struct AVStream'} has no member named 'side_data'
929 | const AVMasteringDisplayMetadata* mastering_display = (const AVMasteringDisplayMetadata*)stream->side_data[i].data;
| ^~~~~~~~~
plugins/video_output.cpp:965:37: error: 'AVStream' {aka 'struct AVStream'} has no member named 'nb_side_data'
965 | for (int i = 0; i < stream->nb_side_data; ++i)
| ^~~~~~~~~~~~
plugins/video_output.cpp:967:25: error: 'AVStream' {aka 'struct AVStream'} has no member named 'side_data'
967 | if (stream->side_data[i].type == AV_PKT_DATA_CONTENT_LIGHT_LEVEL)
| ^~~~~~~~~
plugins/video_output.cpp:969:102: error: 'AVStream' {aka 'struct AVStream'} has no member named 'side_data'
969 | const AVContentLightMetadata* content_light = (const AVContentLightMetadata*)stream->side_data[i].data;

lwindex.c: In function 'get_ticks_per_frame':
lwindex.c:1789:15: error: 'AVCodecContext' has no member named 'ticks_per_frame'
1789 | return ctx->ticks_per_frame;
| ^~
lwlibav_video.c: In function 'field_number_of_picture_in_frame':
lwlibav_video.c:864:14: error: 'AVFrame' has no member named 'top_field_first'
864 | if( frame->top_field_first )
| ^~
lwlibav_video.c: In function 'copy_field':
lwlibav_video.c:1253:8: error: 'AVFrame' has no member named 'interlaced_frame'
1253 | dst->interlaced_frame = src->interlaced_frame;
| ^~
lwlibav_video.c:1253:32: error: 'AVFrame' has no member named 'interlaced_frame'
1253 | dst->interlaced_frame = src->interlaced_frame;
| ^~
lwlibav_video.c:1254:8: error: 'AVFrame' has no member named 'top_field_first'
1254 | dst->top_field_first = top_field_first;
| ^~
lwlibav_video.c: In function 'lwlibav_repeat_control':
lwlibav_video.c:1296:31: error: 'AVFrame' has no member named 'interlaced_frame'
1296 | vdhp->frame_buffer->interlaced_frame = vdhp->last_req_frame->interlaced_frame;
| ^~
lwlibav_video.c:1296:72: error: 'AVFrame' has no member named 'interlaced_frame'
1296 | vdhp->frame_buffer->interlaced_frame = vdhp->last_req_frame->interlaced_frame;
| ^~

[LigH]

The ffmpeg source must have been refactored, many renamed types and constants; e.g. FF_PROFILE~ to AV_PROFILE~ (https://github.com/m-ab-s/media-autobuild_suite/issues/2866)

[Jamaika]

I don't think so. As for the rest, I was rejected from the topic.
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/pull/80

Problem with:
https://github.com/FFmpeg/FFmpeg/commit/4a66fe15880565d006996ac9719a3290b77e9536
https://github.com/FFmpeg/FFmpeg/commit/8af333188349d7eaab141bc872d98e03d03bf029
https://github.com/FFmpeg/FFmpeg/commit/7d07723db5c18bb762f8eeb2a844a677986b8dcc
https://github.com/FFmpeg/FFmpeg/commit/ec8e796b42bccf999fdf31c110385e2cc119b7f2

[VoodooFX]

Here you can try the sneak peak build (https://github.com/user-attachments/files/19379759/LSMASHSource.zip) [before the official build comes out] that supports hardware accelerated decoding for Intel QSV [this one was broken before], d3d11va , dxva2.

prefer_hw=2 / Intel QSV
prefer_hw=4 / dxva2
prefer_hw=5 / d3d11va

EDIT:
Real world encoding benefit [those where short 10000 frames tests] - from ~3% upto 23.4% encoding speedup variation depending on a source file.


Source h264 720p:
dxva2 - 6.3%
d3d11va - 4.9%
Intel QSV - 4.9%

Source h264 1080p:
dxva2 - 2.7%
d3d11va - 3.6%
Intel QSV - 23.4%

[Z2697]

Here you can try the sneak peak build (https://github.com/user-attachments/files/19379759/LSMASHSource.zip) [before the official build comes out] that supports hardware accelerated decoding for Intel QSV [this one was broken before], d3d11va , dxva2.

prefer_hw=2 / Intel QSV
prefer_hw=4 / dxva2
prefer_hw=5 / d3d11va

EDIT:
Real world encoding benefit [those where short 10000 frames tests] - from ~3% upto 23.4% encoding speedup variation depending on a source file.


Source h264 720p:
dxva2 - 6.3%
d3d11va - 4.9%
Intel QSV - 4.9%

Source h264 1080p:
dxva2 - 2.7%
d3d11va - 3.6%
Intel QSV - 23.4%

That's gonna depend on the CPU and encoder settings you use.
A modern CPU can easily do hundreads of frames per second on single thread h264 1080p decoding.
Imagnine you doing the encoding at 10% the single thread decoding speed on a 16C CPU, by switching to HW decode you save 1/16*10% = 0.625% total CPU resource.

[VoodooFX]

That's gonna depend on the CPU and encoder settings you use.
Of course.
Those tests were done on 4 cores 11th Gen Intel CPU.

[Z2697]

Of course.
Those tests were done on 4 cores 11th Gen Intel CPU.

Did you check the GPU decoding engine usage? None of the prefer_hw option works on my system, with new builds. Old builds works fine.

[poisondeathray]

For that test build, prefer_hw=2 does not work for me either, but decoder="h264_qsv" works (when decoding a AVC stream).

Years ago, when both qsv and cuvid worked for old versions of lsmash, I didn't see much benefit for final encoding speed difference . Maybe 1-2% on a 1080 stream. 23.4% seems abnormally high for a 4 core

[VoodooFX]

Did you check the GPU decoding engine usage? None of the prefer_hw option works on my system, with new builds. Old builds works fine.

"prefer_hw=2" in 20240408 1194 doesn't work for me, tested with 2th and 11th Gen CPUs. Same confirmed by Asd-g himself.
What's your CPU?

I didn't check actual GPU usage, but CPU usage is low and LSMASH itself reports that it works on GPU.

This is how you can check if it works:

# Official AvS 3.7.3
LWLibAvVideoSource("D:\xxxxx.mkv", prefer_hw=2)
Subtitle(LWLDECODER)



For that test build, prefer_hw=2 does not work for me either, but decoder="h264_qsv" works (when decoding a AVC stream).

Make sure that r1217 build is in use. What is your CPU?

Years ago, when both qsv and cuvid worked for old versions of lsmash, I didn't see much benefit for final encoding speed difference . Maybe 1-2% on a 1080 stream. 23.4% seems abnormally high for a 4 core

I don't remember if I tested hw with older builds. That looked weird for me too so I repeated the tests multiple times, same results. Encode was running above 100FPS, I think.

[TR-9970X]

Make sure that r1217 build is in use.

I had some weird Avisynth issues with r1217 :(

I dont know what "height" means

r1205 was good :)

After pulling in a new job with RipBot264, which uses Lsmash, when doing a preview of the script in MPC-HC, it showed the above error.

[Z2697]

"prefer_hw=2" in 20240408 1194 doesn't work for me, tested with 2th and 11th Gen CPUs. Same confirmed by Asd-g himself.
What's your CPU?

I didn't check actual GPU usage, but CPU usage is low and LSMASH itself reports that it works on GPU.

This is how you can check if it works:

# Official AvS 3.7.3
LWLibAvVideoSource("D:\xxxxx.mkv", prefer_hw=2)
Subtitle(LWLDECODER)





Make sure that r1217 build is in use. What is your CPU?



I don't remember if I tested hw with older builds. That looked weird for me too so I repeated the tests multiple times, same results. Encode was running above 100FPS, I think.

I only have CUVID available, I disabled the iGPU.

[VoodooFX]

I had some weird Avisynth issues with r1217 :(

I dont know what "height" means

r1205 was good :)


Report it there [with more details than you provided here]: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues

I only have CUVID available, I disabled the iGPU.

Oh, I don't have CUDA, try to report your issue.

[leoenc]

I'm suddenly getting the following error when compiling:
../../common/lwindex.c:58:10: fatal error: xxhash.h: No such file or directory
And the compilation fails.
It was working fine just a few days ago. I've tried apt install xxhash but it didn't help.

Here is the relevant part of my dockerfile (the error is on the last RUN command):
WORKDIR /prereq
RUN git clone https://git.videolan.org/git/ffmpeg.git
WORKDIR /prereq/ffmpeg
RUN mkdir -p /prereq/ffmpeg_build_minimal && \
./configure --prefix=/prereq/ffmpeg_build_minimal \
--enable-gpl --enable-version3 --disable-encoders \
--disable-muxers --disable-doc --disable-debug --enable-libdav1d \
--cpu=native --extra-cflags="-march=native" && \
make -j$(nproc) && \
make install

WORKDIR /prereq
RUN git clone https://github.com/dwbuiten/obuparse.git
WORKDIR /prereq/obuparse
RUN make && \
make install

WORKDIR /prereq
RUN git clone https://github.com/vimeo/l-smash.git
WORKDIR /prereq/l-smash
RUN ./configure --extra-cflags="-march=native -fPIC" && \
make -j$(nproc) && \
checkinstall --pkgname=lsmash --pkgversion="0:$(git rev-list --count HEAD)-g$(git rev-parse --short HEAD)" --backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes --fstrans=no --default

WORKDIR /prereq
RUN git clone https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works.git && \
mkdir -p /prereq/L-SMASH-Works/AviSynth/build
WORKDIR /prereq/L-SMASH-Works/AviSynth/build
RUN PKG_CONFIG_PATH=/prereq/ffmpeg_build_minimal/lib/pkgconfig LDFLAGS="-Wl,-Bsymbolic" meson setup .. && \
ninja && \
checkinstall --pkgname=lsmashsource --pkgversion="1:$(git rev-list --count HEAD)-g$(git rev-parse --short HEAD)" --backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes --stripso=yes --addso=yes --fstrans=no --default ninja install

[StvG]

xxHash is submodule (https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/commit/42157a2c2c2aef5c548295dbf985fcaff36ac27e).
Either change RUN git clone https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works.git to RUN git clone --recurse-submodules https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works.git and change building to the official one - with CMake.
Or change in lwindex.c #include "xxhash.h" to #include <xxhash.h> in order to use the system xxHash you installed.

[Xuqiwu]

Looks like Asd-g and co is close to release a new version.

So he has made new test builds 2 days ago.

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/94#issuecomment-2799927594

[LigH]

I noticed that MPEG-1 video gets decoded with doubled frames in field rate, e.g. NTSC 352x240 with 29.97 fps becomes 59.94 fps with identical odd and even frames. Can't tell for sure yet, not much tested so far.

[Katie Boundary]

Does an XP-compatible version of this exist? FFmpegsource is having some problems decoding the audio from one of my files.

[flossy_cake]

Is it still recommended to call LwLibavAudioSource BEFORE video source to avoid double indexing or did that get optimised in a update? I seem to recall Asd-g did an update relating to that, and some other memory where I was testing it and there was no speedup by doing the audio first

[rgr]

Is it still recommended to call LwLibavAudioSource BEFORE video source to avoid double indexing or did that get optimised in a update? I seem to recall Asd-g did an update relating to that, and some other memory where I was testing it and there was no speedup by doing the audio first

Last time I checked, yes. I had two indexing messages. But maybe that's changed recently. Regardless, an LWLibavSource function, like in ffms2 or BestSource, would be useful.

[flossy_cake]

Last time I checked, yes. I had two indexing messages. But maybe that's changed recently.

I just tested now with ffmpeg and yes I also see 1 indexing message in the console when doing audio first , and 2 when doing video first

But there is some optimisation going on which hides the performance gain of doing audio first: if i do the video source first , the second indexing message goes superfast like 0-100% in 2 seconds for a 2 hour movie. Maybe the video indexing which was completed beforehand is already in memory/pagefile , or possibly LwLibAVAudioSource is looking inside the index file which LWLibAvVideoSource created and seeing it already contains video index and skips doing a full index a second time and just does some kind of "quick check" mode?


edit: also wouldn't this depend on which order the Avisynth interpreter references video/audio in the call to AudioDub(video, audio)? The interpreter must be doing it backwards and referencing the second argument "audio" first otherwise "video" points to LWLibAvVideoSource and would call that first - is this correct?

edit2: no that can't be right because the appearance of 2 indexing messages depends on the order in which I do video= and audio= BEFORE the AudioDub. So the interpreter must be evaluating video= and audio= upstream from AudioDub. Coulda sworn I read something in the wiki that says the interpreter works its way backwards through the script or something like that

[wonkey_monkey]

if i do the video source first , the second indexing message goes superfast like 0-100% in 2 seconds for a 2 hour movie. Maybe the video indexing which was completed beforehand is already in memory/pagefile

Or: the whole video file has now been cached by the OS in RAM. I had a utility which would read a file from start to end, discarding the contents itself, but it left the file cached in RAM and subsequently loading it into VirtualDub and scrubbing through it was much quicker.

[flossy_cake]

Or: the whole video file has now been cached by the OS in RAM. I had a utility which would read a file from start to end, discarding the contents itself, but it left the file cached in RAM and subsequently loading it into VirtualDub and scrubbing through it was much quicker.

Yeah that must be it.

But what I don't understand is why it's still fast the second time even after I delete the .lwi file. Because that's a new file creation and the OS doesn't know anything about the contents of that file yet.

Even if I rename the source video so that the generated .lwi file is a new name, it's still fast!

I have to actually reboot my whole PC to remove it from the OS cache and get the real "first time" slow speed

[LigH]

Even if you rename your source file, the OS knows it is still the same file which still has the same position on disk. It has a mapping between file areas on your disk and cache areas in your RAM.

[rgr]

I have to actually reboot my whole PC to remove it from the OS cache and get the real "first time" slow speed

LWLibavVideoSource has slowed down recently as it examines more data. There's probably an option to speed it up, and it should be on GitHub with one of the releases.

[hello_hello]

Even if I rename the source video so that the generated .lwi file is a new name, it's still fast!
I have to actually reboot my whole PC to remove it from the OS cache and get the real "first time" slow speed

What if you rename the file and move it to a new folder/location?

Or if it's faster than rebooting, you could remux it with something like MKVToolNix so it's a new file.

[flossy_cake]

What if you rename the file and move it to a new folder/location?

Still lighting fast.

But I get it now - the bottleneck is in the READING of the source file off the disk, not the CPU processing of the indexing process, nor the WRITING of the index file to disk.

My reasoning: A DVD movie is approximately 5GB which if read from disk at around 180MB/sec = about 27 seconds , which is consistent with the indexing time.

Then after indexing is complete , that 5GB movie is still sitting in system RAM so next time LW indexer needs it OS can deliver it straight from RAM and in this case I see CPU core is fully loaded up now (albeit single threaded) whereas when reading it off the disk the first time the CPU is barely doing anything (this also wrongfully lead me to suspect multithreading could speed up indexing)

[Atak_Snajpera]

Still lighting fast.

But I get it now - the bottleneck is in the READING of the source file off the disk, not the CPU processing of the indexing process, nor the WRITING of the index file to disk.

My reasoning: A DVD movie is approximately 5GB which if read from disk at around 180MB/sec = about 27 seconds , which is consistent with the indexing time.

Then after indexing is complete , that 5GB movie is still sitting in system RAM so next time LW indexer needs it OS can deliver it straight from RAM and in this case I see CPU core is fully loaded up now (albeit single threaded) whereas when reading it off the disk the first time the CPU is barely doing anything (this also wrongfully lead me to suspect multithreading could speed up indexing)

Yes you are right. You can easily check what files are already cached in RAM with this cool tool
https://learn.microsoft.com/en-us/sysinternals/downloads/rammap

[flossy_cake]

Yes you are right. You can easily check what files are already cached in RAM with this cool tool
https://learn.microsoft.com/en-us/sysinternals/downloads/rammap

Thanks but I can't look at it cause I've blocked all microsoft domains at router level as a precaution against auto updates of the OS :devil:

Anyway the moral of the story: you don't need to do audio indexing first cause OS will deliver it out of RAM - BUT - there are other media files that take longer to index like say a movie with a 1500kbps 5.1 DTS audio track, the indexing takes way longer for that so still ultimately better to do the Audio first as a general rule or "best practices" type thing. For a typical 2 hour DVD movie though it's only about 5 seconds extra to do it out of RAM a second time.

[flossy_cake]

Is there any possibility that audio track selection by language could be added?

LAV video decoder for DirectShow supports it - wouldn't this mean language detection is built into libav itself and would be trivial to implement?

Anyway here is a use case scenario: Looney Tunes collection (800GB) contains hundreds of mkv files, each with multiple language tracks and the English track is on a different track number for each file. So selecting by track number is not an option unless you want to manually inspect each file.

Currently: LWLibavAudioSource(stream_index=1)

Proposed: LWLibavAudioSource(stream_lang="eng")

:thanks:

edit: well, there is a theoretical way to work around it: using DirectShowSource() for the audio track (which invokes nevcairiel's DirectShow LAV splitter in the system tray, which handles the language selection). But DirectShowSource() can't handle non-English characters in file paths (which many files in this large collection contains) so I'd still need to manually inspect/modify each file. Plus DirectShowSource is kind of flakey and I'd much prefer to use libav. Kinda crazy after decades of Avisynth development there is still not a source filter that can select audio track by language.

edit2: DirectShowSource() is indeed janky with audio sync issues, seem to need EnsureVBRMP3Sync() to fix it from being up to 10 seconds out of sync, but reliability is questionable.

edit3: and in the LAV Splitter instance in the system tray that was invoked by DirectShowSource(), untick "Prefer audio streams with highest quality" to force it to choose the first english track rather than the highest quality english track (avoids some false positives where the highest quality english track was actually a commentary track).

edit4: and to clean the file path of foreign language characters: cleanedFilePath = RegExReplace(filePath, "[^\x20-\x7E]", "") (replaces any characters outside the range ASCII 32 to 126 with empty string: "")

[StainlessS]

Thanks but I can't look at it cause I've blocked all microsoft domains at router level as a precaution against auto updates of the OS :devil:

Flossy, above, older post but maybe you wanna try this,

https://www.sordum.org/9470/windows-update-blocker-v1-8/

https://i.postimg.cc/MHH153X8/Untitled-01.jpg (https://postimages.org/)

Easy Peasy to switch on/off, been using it for several years.
[I only ever select the "enable/disable updates" radio buttons and then apply.]

EDIT: non-install, run it directly.

EDIT: Maybe its because of telemetry that you block all M$ domains, not necessary just to block Auto Update.

[flossy_cake]

Flossy, above, older post but maybe you wanna try this,

https://www.sordum.org/9470/windows-update-blocker-v1-8/

https://i.postimg.cc/MHH153X8/Untitled-01.jpg (https://postimages.org/)

Easy Peasy to switch on/off, been using it for several years.
[I only ever select the "enable/disable updates" radio buttons and then apply.]

EDIT: non-install, run it directly.

EDIT: Maybe its because of telemetry that you block all M$ domains, not necessary just to block Auto Update.


Cool, looks like it works by blocking the Windows update service(s).

Of course, the first thing I tried ages ago was disabling those services and they just kept getting automatically re-enabled by Windows. So that app must be polling the service running state every n seconds or so and saying "no ya dont" :devil:

The way I'm currently blocking updates is with a bunch of regkeys and a group policy setting (the latter of which only works on the Pro version since Home has no group policy editor) and that resulted in the update screen looking like this now:

https://i.imgur.com/FNx4Rdx.png


https://i.imgur.com/f7y5EZK.png


Clicking "check for updates" would still update, so I needed to block all these domains at my router too:


https://i.imgur.com/hE4XsgU.png


After blocking those domains, now when I click "check for updates" I just get the animating spinner for eternity and it never updates.

However it's probably wise to also block the update services with that app as well because for all I know Windows could force click the "check for updates" button click at some point, in which case my only line of defense would be my router, and I'd like at least 2 lines of defense.

Another line of defense is to use Clonezilla to image the OS drive so I can roll back if the worst happens.

Looks like I'll be using Windows 10 for the next 10 years or so, I can't think of anything I would need 11 for.

[StainlessS]

I have not ever found it necessary to use any of the more 'advanced' options of WindowsUpdateBlocker, it aint ever failed for me.

But could additionally try "WindowsUpdatePause.reg" from here:-
https://forum.doom9.org/showthread.php?p=2014221#post2014221

Yes, always a good idea to have an image of OS (or several images including initial setup with then current updates, perhaps updated to W10 Final updates).

[flossy_cake]

Is there any possibility that audio track selection by language could be added?
Currently: LWLibavAudioSource(stream_index=1)
Proposed: LWLibavAudioSource(stream_lang="eng")
edit: well, there is a theoretical way to work around it

I will post the solution here for how to open audio track by language, AND also render subtitle track by language!

Note: you will need the Directshow LAV and and DirectVobSub installed (recommend installing k-lite codec pack for this)

source = "C:\MyVideo.mkv"

video = DSS2(source, /* http://www.avisynth.nl/index.php/DSS2mod */
\
\ subsm=2, /* force loading of DirectVobSub to the graph for subtitle rendering.
\ Select rendering style options through the DirectVobSub tray icon. */
\
\ lavs="l3 sa[*:eng|f]", /* sa[*:eng] = choose first eng sub
\ sa[*:eng|f] = choose eng "forced" sub eg. sign subs only
\ sa[eng:off; *:eng] = choose eng sub only when audio track is not eng */
\
\ lavd="l3 d1", /* d0=ordered dithering, d1=random dithering.
\ Needed for 10 bit videos since DSS2 downconverts to 8-bit - doh!
\ Verified with "gradient_1920-1080_0-25_yuv420p10_x265.mp4" on github */
\
\ lavf_path="C:\Program Files (x86)\K-Lite Codec Pack\Filters\LAV64" )


try {
audio = DirectShowSource(source,
\
\ video=false,
\
\ audio=true /* Make audio track selection rules through the LAV Splitter tray icon, eg.
\ type "eng" in audio field, and choose "prefer stream with highest quality"
\ if needed eg. when there are multiple eng tracks. Note: there will be
\ multiple instances of LAV splitters in the tray if using MPC-HC, so make
\ sure to identify the correct one belonging to DirectShowSource().
\ On my system it's coloured differently to the MPC-HC instance. */
\ )

AudioDub(video, audio)
EnsureVBRMP3Sync() /* Needed for audio sync in some files */
}
catch (noAudio) { video }


Bonus: for NTSC soft telecine files like DVD remuxes, use DSS2(fps=59.940).SelectEven() which should
produce 30p 1:1:1:2 cadence like you would normally get out of TFM for hard telecine files, then you can
TDecimate() the 1 in 5 to 24p as usual.

:thanks:


edit: tip - make sure to check and understand all the LAV audio decoder settings (via the system tray icon). Especially "normalize matrix" should be turned on when Directshow LAV is downmixing surround to stereo to avoid clipping and then Normalise() in Avisynth afterwards to regain lost volume caused by LAV's downmixing.

[hello_hello]

flossy_cake,

This isn't DirectShow related, but if you don't mind encoding the audio separately, foobar2000 can be a good option for downmixing and re-encoding.
It can open most file types, but there's also a 32 bit AVS plugin for the 32 bit foobar2000 (which means you need to have 32 bit Avisynth installed) so you can downmix in Avisynth and then scan the output of the script to check the peak level.

The audio in this example was opened in a script with FFMS2 (although DirectShowSource should work just as well). It's from a 45 minute episode of a TV show. As you can see at the bottom of the scan result window, it took FB2K about 7.5 seconds to scan the script output. Reducing the volume by 2dB in the script was enough to prevent clipping.

# Downmixing 5.1ch

KillVideo() # FB2k will likely crash if the script outputs video.

Audio = last.ConvertAudioToFloat()

FL = GetChannel(Audio, 1)
FR = GetChannel(Audio, 2)
FC = GetChannel(Audio, 3)
LFE = GetChannel(Audio, 4) # Not included in downmix
SL = GetChannel(Audio, 5)
SR = GetChannel(Audio, 6)

FrontGain = 1.0
CentreGain = 0.7079457
SurroundGain = 0.7079457

# Mix Front Left/Right & Surround Left/Right

FL_SL = MixAudio(FL, SL, FrontGain, SurroundGain)
FR_SR = MixAudio(FR, SR, FrontGain, SurroundGain)

# Add Centre

Left = MixAudio(FL_SL, FC, 1.0, CentreGain)
Right = MixAudio(FR_SR, FC, 1.0, CentreGain)

# Merge channels

Stereo = MergeChannels(Left, Right)

# Adjust volume if necessary

Stereo = Stereo.AmplifydB(-2)

return Stereo

https://i.imgur.com/K04zYmk.png

And of course you can use FB2K's converter to convert it.
It took about 15 seconds to convert my 45 minute test script to AAC with QAAC, and this was the result after scanning it.

https://i.imgur.com/ACCmLy3.png

[flossy_cake]

flossy_cake,

This isn't DirectShow related, but if you don't mind encoding the audio separately, foobar2000 can be a good option for downmixing and re-encoding.

Neat.

And let's not forget Avisynth LAV can downmix too with eg. LwLibavAudioSource(layout="stereo").Normalise(0.98)

99% of my Avisynth usage is for realtime use watching stuff through MPC-HC so I don't usually do any transcoding.

I did have an audio transcoding issue recently where my father's bluray player wouldn't decode DTS from an mkv file (but could from bluray discs) so I had to transcode to AAC and then I found out there are other issues to do with 5.1 layout like AAC defaulting to hexagonal layout or whatever it was, I already forgot what the solution was lol why does it all have to be so technical like this? :devil: