Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th February 2019, 18:29   #1  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Indexing m3u8 file

Hi,
I have a .m3u8 metadata file which has metadata about many videos to append to make a single file.
I tried to index it using FFVideoSource, LWLibavVideoSource and DirectShowSource, but none of them recognized the .m3u8 metadata file.
I could simply index the .ts files individually and then append them together, but this is simply not practical.

Code:
LWLibavVideoSource("Z:\Distribution\segment_0.ts")
segment0=last

LWLibavVideoSource("Z:\Distribution\segment_1.ts")
segment1=last

LWLibavVideoSource("Z:\Distribution\segment_2.ts")
segment2=last

LWLibavVideoSource("Z:\Distribution\segment_3.ts")
segment3=last

LWLibavVideoSource("Z:\Distribution\segment_4.ts")
segment4=last

LWLibavVideoSource("Z:\Distribution\segment_5.ts")
segment5=last

LWLibavVideoSource("Z:\Distribution\segment_6.ts")
segment6=last

LWLibavVideoSource("Z:\Distribution\segment_7.ts")
segment7=last

LWLibavVideoSource("Z:\Distribution\segment_8.ts")
segment8=last

LWLibavVideoSource("Z:\Distribution\segment_9.ts")
segment9=last

LWLibavVideoSource("Z:\Distribution\segment_10.ts")
segment10=last

LWLibavVideoSource("Z:\Distribution\segment_11.ts")
segment11=last

LWLibavVideoSource("Z:\Distribution\segment_12.ts")
segment12=last

LWLibavVideoSource("Z:\Distribution\segment_13.ts")
segment13=last

LWLibavVideoSource("Z:\Distribution\segment_14.ts")
segment14=last

LWLibavVideoSource("Z:\Distribution\segment_15.ts")
segment15=last

LWLibavVideoSource("Z:\Distribution\segment_16.ts")
segment16=last

LWLibavVideoSource("Z:\Distribution\segment_17.ts")
segment17=last

LWLibavVideoSource("Z:\Distribution\segment_18.ts")
segment18=last

LWLibavVideoSource("Z:\Distribution\segment_19.ts")
segment19=last

LWLibavVideoSource("Z:\Distribution\segment_20.ts")
segment20=last

LWLibavVideoSource("Z:\Distribution\segment_21.ts")
segment21=last

LWLibavVideoSource("Z:\Distribution\segment_22.ts")
segment22=last

LWLibavVideoSource("Z:\Distribution\segment_23.ts")
segment23=last

LWLibavVideoSource("Z:\Distribution\segment_24.ts")
segment24=last

LWLibavVideoSource("Z:\Distribution\segment_25.ts")
segment25=last

LWLibavVideoSource("Z:\Distribution\segment_26.ts")
segment26=last

LWLibavVideoSource("Z:\Distribution\segment_27.ts")
segment27=last

LWLibavVideoSource("Z:\Distribution\segment_28.ts")
segment28=last

LWLibavVideoSource("Z:\Distribution\segment_29.ts")
segment29=last

LWLibavVideoSource("Z:\Distribution\segment_30.ts")
segment30=last

LWLibavVideoSource("Z:\Distribution\segment_31.ts")
segment31=last

LWLibavVideoSource("Z:\Distribution\segment_32.ts")
segment32=last

LWLibavVideoSource("Z:\Distribution\segment_33.ts")
segment33=last

LWLibavVideoSource("Z:\Distribution\segment_34.ts")
segment34=last

LWLibavVideoSource("Z:\Distribution\segment_35.ts")
segment35=last

LWLibavVideoSource("Z:\Distribution\segment_36.ts")
segment36=last

LWLibavVideoSource("Z:\Distribution\segment_37.ts")
segment37=last

LWLibavVideoSource("Z:\Distribution\segment_38.ts")
segment38=last

LWLibavVideoSource("Z:\Distribution\segment_39.ts")
segment39=last

LWLibavVideoSource("Z:\Distribution\segment_40.ts")
segment40=last

LWLibavVideoSource("Z:\Distribution\segment_41.ts")
segment41=last

LWLibavVideoSource("Z:\Distribution\segment_42.ts")
segment42=last

LWLibavVideoSource("Z:\Distribution\segment_43.ts")
segment43=last

LWLibavVideoSource("Z:\Distribution\segment_44.ts")
segment44=last

LWLibavVideoSource("Z:\Distribution\segment_45.ts")
segment45=last

LWLibavVideoSource("Z:\Distribution\segment_46.ts")
segment46=last

LWLibavVideoSource("Z:\Distribution\segment_47.ts")
segment47=last

LWLibavVideoSource("Z:\Distribution\segment_48.ts")
segment48=last

LWLibavVideoSource("Z:\Distribution\segment_49.ts")
segment49=last

LWLibavVideoSource("Z:\Distribution\segment_50.ts")
segment50=last

LWLibavVideoSource("Z:\Distribution\segment_51.ts")
segment51=last

LWLibavVideoSource("Z:\Distribution\segment_52.ts")
segment52=last

LWLibavVideoSource("Z:\Distribution\segment_53.ts")
segment53=last

LWLibavVideoSource("Z:\Distribution\segment_54.ts")
segment54=last

LWLibavVideoSource("Z:\Distribution\segment_55.ts")
segment55=last

LWLibavVideoSource("Z:\Distribution\segment_56.ts")
segment56=last

LWLibavVideoSource("Z:\Distribution\segment_57.ts")
segment57=last

LWLibavVideoSource("Z:\Distribution\segment_58.ts")
segment58=last

LWLibavVideoSource("Z:\Distribution\segment_59.ts")
segment59=last

LWLibavVideoSource("Z:\Distribution\segment_60.ts")
segment60=last

LWLibavVideoSource("Z:\Distribution\segment_61.ts")
segment61=last

LWLibavVideoSource("Z:\Distribution\segment_62.ts")
segment62=last

LWLibavVideoSource("Z:\Distribution\segment_63.ts")
segment63=last

LWLibavVideoSource("Z:\Distribution\segment_64.ts")
segment64=last

LWLibavVideoSource("Z:\Distribution\segment_65.ts")
segment65=last

LWLibavVideoSource("Z:\Distribution\segment_66.ts")
segment66=last

LWLibavVideoSource("Z:\Distribution\segment_67.ts")
segment67=last

LWLibavVideoSource("Z:\Distribution\segment_68.ts")
segment68=last

LWLibavVideoSource("Z:\Distribution\segment_69.ts")
segment69=last

LWLibavVideoSource("Z:\Distribution\segment_70.ts")
segment70=last

LWLibavVideoSource("Z:\Distribution\segment_71.ts")
segment71=last

part1=segment1++segment2++segment3++segment4++segment5++segment6++segment7++segment8++segment9++segment10
part2=segment11++segment12++segment13++segment14++segment15++segment16++segment17++segment18++segment19++segment20
part3=segment21++segment22++segment23++segment24++segment25++segment26++segment27++segment28++segment29++segment30
part4=segment31++segment32++segment33++segment34++segment35++segment36++segment37++segment38++segment39++segment40
part5=segment41++segment42++segment43++segment44++segment45++segment46++segment47++segment48++segment49++segment50
part6=segment51++segment52++segment53++segment54++segment55++segment56++segment57++segment58++segment59++segment60
part7=segment61++segment62++segment63++segment64++segment65++segment66++segment67++segment68++segment69++segment70
part8=segment71

encode=part1++part2++part3++part4++part5++part6++part7++part8

return encode

Basically the .m3u8 file does what I wrote above and it would be way easier and faster.
Any ideas?
__________________
Broadcast Encoder
LinkedIn
FranceBB is offline   Reply With Quote
Old 25th February 2019, 20:06   #2  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 758
Load all the .ts files at once in DGIndexNV and save project.
videoh is offline   Reply With Quote
Old 25th February 2019, 20:24   #3  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Quote:
Originally Posted by videoh View Post
Load all the .ts files at once in DGIndexNV and save project.
I opened DGAVCIndex and I loaded all the files. (File -> Open).
I now have a file list of 71 files, but if I try to save the project [F4], it only indexes Segment 0.
__________________
Broadcast Encoder
LinkedIn
FranceBB is offline   Reply With Quote
Old 25th February 2019, 21:16   #4  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 758
DGAVCIndex is not DGIndexNV. The streams probably have EOS NALUs. If I remember correctly, DGIndexNV ignores them, but DGAVCIndex may not.
videoh is offline   Reply With Quote
Old 25th February 2019, 22:52   #5  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Quote:
Originally Posted by videoh View Post
DGAVCIndex is not DGIndexNV. The streams probably have EOS NALUs. If I remember correctly, DGIndexNV ignores them, but DGAVCIndex may not.
I know, DGIndexNV is for paying users, is more updated and it uses Nvidia Video Cards via CUDA, but the reason why I never downloaded it it's because not all my computers have an NVIDIA GPU, as some of them are Linux boxes running Virtual Machines or dedicated servers without a GPU.
However, if it can run via software as well (CPU only), I will definitely switch from DGIndex/DGAVCIndex to DGIndexNV.
__________________
Broadcast Encoder
LinkedIn
FranceBB is offline   Reply With Quote
Old 25th February 2019, 23:14   #6  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 758
DGDecNV requires an nVidia GPU.
videoh is offline   Reply With Quote
Old 26th February 2019, 01:03   #7  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,049
Quote:
Originally Posted by FranceBB View Post
Hi,
I have a .m3u8 metadata file which has metadata about many videos to append to make a single file.
I tried to index it using FFVideoSource, LWLibavVideoSource and DirectShowSource, but none of them recognized the .m3u8 metadata file.
I could simply index the .ts files individually and then append them together, but this is simply not practical.

Code:
LWLibavVideoSource("Z:\Distribution\segment_0.ts")
segment0=last

LWLibavVideoSource("Z:\Distribution\segment_1.ts")
segment1=last

LWLibavVideoSource("Z:\Distribution\segment_2.ts")
segment2=last

LWLibavVideoSource("Z:\Distribution\segment_3.ts")
segment3=last

LWLibavVideoSource("Z:\Distribution\segment_4.ts")
segment4=last

LWLibavVideoSource("Z:\Distribution\segment_5.ts")
segment5=last

LWLibavVideoSource("Z:\Distribution\segment_6.ts")
segment6=last

LWLibavVideoSource("Z:\Distribution\segment_7.ts")
segment7=last

LWLibavVideoSource("Z:\Distribution\segment_8.ts")
segment8=last

LWLibavVideoSource("Z:\Distribution\segment_9.ts")
segment9=last

LWLibavVideoSource("Z:\Distribution\segment_10.ts")
segment10=last

LWLibavVideoSource("Z:\Distribution\segment_11.ts")
segment11=last

LWLibavVideoSource("Z:\Distribution\segment_12.ts")
segment12=last

LWLibavVideoSource("Z:\Distribution\segment_13.ts")
segment13=last

LWLibavVideoSource("Z:\Distribution\segment_14.ts")
segment14=last

LWLibavVideoSource("Z:\Distribution\segment_15.ts")
segment15=last

LWLibavVideoSource("Z:\Distribution\segment_16.ts")
segment16=last

LWLibavVideoSource("Z:\Distribution\segment_17.ts")
segment17=last

LWLibavVideoSource("Z:\Distribution\segment_18.ts")
segment18=last

LWLibavVideoSource("Z:\Distribution\segment_19.ts")
segment19=last

LWLibavVideoSource("Z:\Distribution\segment_20.ts")
segment20=last

LWLibavVideoSource("Z:\Distribution\segment_21.ts")
segment21=last

LWLibavVideoSource("Z:\Distribution\segment_22.ts")
segment22=last

LWLibavVideoSource("Z:\Distribution\segment_23.ts")
segment23=last

LWLibavVideoSource("Z:\Distribution\segment_24.ts")
segment24=last

LWLibavVideoSource("Z:\Distribution\segment_25.ts")
segment25=last

LWLibavVideoSource("Z:\Distribution\segment_26.ts")
segment26=last

LWLibavVideoSource("Z:\Distribution\segment_27.ts")
segment27=last

LWLibavVideoSource("Z:\Distribution\segment_28.ts")
segment28=last

LWLibavVideoSource("Z:\Distribution\segment_29.ts")
segment29=last

LWLibavVideoSource("Z:\Distribution\segment_30.ts")
segment30=last

LWLibavVideoSource("Z:\Distribution\segment_31.ts")
segment31=last

LWLibavVideoSource("Z:\Distribution\segment_32.ts")
segment32=last

LWLibavVideoSource("Z:\Distribution\segment_33.ts")
segment33=last

LWLibavVideoSource("Z:\Distribution\segment_34.ts")
segment34=last

LWLibavVideoSource("Z:\Distribution\segment_35.ts")
segment35=last

LWLibavVideoSource("Z:\Distribution\segment_36.ts")
segment36=last

LWLibavVideoSource("Z:\Distribution\segment_37.ts")
segment37=last

LWLibavVideoSource("Z:\Distribution\segment_38.ts")
segment38=last

LWLibavVideoSource("Z:\Distribution\segment_39.ts")
segment39=last

LWLibavVideoSource("Z:\Distribution\segment_40.ts")
segment40=last

LWLibavVideoSource("Z:\Distribution\segment_41.ts")
segment41=last

LWLibavVideoSource("Z:\Distribution\segment_42.ts")
segment42=last

LWLibavVideoSource("Z:\Distribution\segment_43.ts")
segment43=last

LWLibavVideoSource("Z:\Distribution\segment_44.ts")
segment44=last

LWLibavVideoSource("Z:\Distribution\segment_45.ts")
segment45=last

LWLibavVideoSource("Z:\Distribution\segment_46.ts")
segment46=last

LWLibavVideoSource("Z:\Distribution\segment_47.ts")
segment47=last

LWLibavVideoSource("Z:\Distribution\segment_48.ts")
segment48=last

LWLibavVideoSource("Z:\Distribution\segment_49.ts")
segment49=last

LWLibavVideoSource("Z:\Distribution\segment_50.ts")
segment50=last

LWLibavVideoSource("Z:\Distribution\segment_51.ts")
segment51=last

LWLibavVideoSource("Z:\Distribution\segment_52.ts")
segment52=last

LWLibavVideoSource("Z:\Distribution\segment_53.ts")
segment53=last

LWLibavVideoSource("Z:\Distribution\segment_54.ts")
segment54=last

LWLibavVideoSource("Z:\Distribution\segment_55.ts")
segment55=last

LWLibavVideoSource("Z:\Distribution\segment_56.ts")
segment56=last

LWLibavVideoSource("Z:\Distribution\segment_57.ts")
segment57=last

LWLibavVideoSource("Z:\Distribution\segment_58.ts")
segment58=last

LWLibavVideoSource("Z:\Distribution\segment_59.ts")
segment59=last

LWLibavVideoSource("Z:\Distribution\segment_60.ts")
segment60=last

LWLibavVideoSource("Z:\Distribution\segment_61.ts")
segment61=last

LWLibavVideoSource("Z:\Distribution\segment_62.ts")
segment62=last

LWLibavVideoSource("Z:\Distribution\segment_63.ts")
segment63=last

LWLibavVideoSource("Z:\Distribution\segment_64.ts")
segment64=last

LWLibavVideoSource("Z:\Distribution\segment_65.ts")
segment65=last

LWLibavVideoSource("Z:\Distribution\segment_66.ts")
segment66=last

LWLibavVideoSource("Z:\Distribution\segment_67.ts")
segment67=last

LWLibavVideoSource("Z:\Distribution\segment_68.ts")
segment68=last

LWLibavVideoSource("Z:\Distribution\segment_69.ts")
segment69=last

LWLibavVideoSource("Z:\Distribution\segment_70.ts")
segment70=last

LWLibavVideoSource("Z:\Distribution\segment_71.ts")
segment71=last

part1=segment1++segment2++segment3++segment4++segment5++segment6++segment7++segment8++segment9++segment10
part2=segment11++segment12++segment13++segment14++segment15++segment16++segment17++segment18++segment19++segment20
part3=segment21++segment22++segment23++segment24++segment25++segment26++segment27++segment28++segment29++segment30
part4=segment31++segment32++segment33++segment34++segment35++segment36++segment37++segment38++segment39++segment40
part5=segment41++segment42++segment43++segment44++segment45++segment46++segment47++segment48++segment49++segment50
part6=segment51++segment52++segment53++segment54++segment55++segment56++segment57++segment58++segment59++segment60
part7=segment61++segment62++segment63++segment64++segment65++segment66++segment67++segment68++segment69++segment70
part8=segment71

encode=part1++part2++part3++part4++part5++part6++part7++part8

return encode

Basically the .m3u8 file does what I wrote above and it would be way easier and faster.
Any ideas?
Just cleaning up that syntax would be faster, to start with. Using variables and nesting variables together for no script-obvious reason¹ sucks up a lot of time if you're doing all of that manually (typing or copy-paste, it doesn't matter). And if you are doing it through an automatic process, there are better options to generate cleaner scripts for that, too. A one-line for loop in Bash could generate the AviSynth script to do the exact same task as the provided example, and do it in less than a second (or at least a few seconds):
Code:
for n in *.mkv ; do echo "LwLibavSource(\""$n"\") ++ \\" ; done  | sed '$s, ++ \\,,'> output.avs
For FFMS2 at least, the indexing and generating the script can be done in two lines:
Code:
for n in *.mkv ; do ffmsindex -t -1 "$n" ; done

for n in *.mkv ; do echo "FFmpegSource2(\""$n"\",atrack=-1) ++ \\" ; done  | sed '$s, ++ \\,,'> output.avs
The only caveat being that the number increments in the filenames need to be 0-padded so it orders them correctly.


¹ Since there's no indication that the 'part' variables are going to be filtered with differing parameters (or filtered at all), those serve no purpose except a certain style of readability. 'encode', likewise, isn't being chosen from among other very large variables with different combinations of inputs, so there's no point toward defining it as the concatenation of all those part segments, just to turn around and explicitly return 'encode'. It's a waste of space to also assign the 'segment' variables after the source invocation while using an explicit last, rather than simply assigning the source directly into the variable. And if there is no filtering to do on any of it, and it's purely just being appended together, then none of the variables are necessary at all.

In honesty, I wouldn't try to run the .m3u8 through AviSynth, or to try to keep the .ts files separate if the end result is just to have them concatenated anyway. I'd combine them using FFmpeg first, and then just use the single input as source to AviSynth rather than 71 of them. And FFmpeg understands *.m3u8 files just fine:
Code:
ffmpeg -i input.m3u8 -vcodec copy -acodec copy -bsf:a aac_adtstoasc output.mp4
(or whatever your desired output container is)

Last edited by qyot27; 26th February 2019 at 01:11.
qyot27 is offline   Reply With Quote
Old 26th February 2019, 08:16   #8  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,135
While we're at it, I just want to report that DGIndexNV output the wrong color in vapoursynth on my dvd video, while no problem in avisynth.
lansing is offline   Reply With Quote
Old 26th February 2019, 13:54   #9  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 758
Quote:
Originally Posted by lansing View Post
While we're at it, I just want to report that DGIndexNV output the wrong color in vapoursynth on my dvd video, while no problem in avisynth.
If you provide a stream and your script etc. I can look into it. Your post here is too vague to do anything about. It's not clear that we don't have simple pilot error. You can open a thread at my forum too if you prefer.
videoh is offline   Reply With Quote
Old 26th February 2019, 17:04   #10  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Quote:
Originally Posted by qyot27 View Post
In honesty, I wouldn't try to run the .m3u8 through AviSynth, or to try to keep the .ts files separate if the end result is just to have them concatenated anyway. I'd combine them using FFmpeg first, and then just use the single input as source to AviSynth rather than 71 of them. And FFmpeg understands *.m3u8 files just fine:
Code:
ffmpeg -i input.m3u8 -vcodec copy -acodec copy -bsf:a aac_adtstoasc output.mp4
(or whatever your desired output container is)
Got it.
I guess you are right.
I'm running them through ffmpeg with the following bat file as it's so much easier:

Code:
@echo off
setlocal disableDelayedExpansion
::
:: first assume normal call, get args from %*
set args=%*
set "dragDrop="
::
:: Now check if drag&drop situation by looking for %0 in !cmdcmdline!
:: if found then set drag&drop flag and get args from !cmdcmdline!
setlocal enableDelayedExpansion
set "cmd=!cmdcmdline!"
set "cmd2=!cmd:*%~f0=!"
if "!cmd2!" neq "!cmd!" (
  set dragDrop=1
  set "args=!cmd2:~0,-1! "
  set "args=!args:* =!"
)
::
:: Process the first argument only
for %%F in (!args!) do (
  if "!!"=="" endlocal & set "dragDrop=%dragDrop%"
  rem ------------------------------------------------
  rem - Your file processing starts here.
  rem - Use %%F wherever you would normally use %1
  rem
  rem Change to drive and directory of input file
  %%~dF
  cd %%~pF
  C:\Encoding\ffmpeg.exe -hwaccel dxva2 -i "%%~nxF" -threads 32 -vcodec copy -acodec copy "C:\Encoding\%%~nF output.mp4" 
  rem
  rem - Your file processing ends here
  rem -------------------------------------------------
)
if defined dragDrop (
  pause
  exit
)
__________________
Broadcast Encoder
LinkedIn
FranceBB is offline   Reply With Quote
Old 26th February 2019, 17:30   #11  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,135
Quote:
Originally Posted by videoh View Post
If you provide a stream and your script etc. I can look into it. Your post here is too vague to do anything about. It's not clear that we don't have simple pilot error. You can open a thread at my forum too if you prefer.
Here's the sample clip, the red color is off when opening in vapoursynth.

Okay enough hijacking for me.
lansing is offline   Reply With Quote
Old 26th February 2019, 19:15   #12  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 758
Thanks for not providing your script, telling me why you think it is "off", etc.

Last edited by videoh; 26th February 2019 at 19:18.
videoh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.