L-SMASH Source

[l33tmeatwad]

I just tested it by 32-bit AVSMeter on a few clips with codecs avc hevc vc-1 mpeg2 vp9 av1, and I couldn't reproduce it.
Tried it on two different Windows 10 machines (both 64-bit Windows) and on macOS though WINE. The x64 version works on the same machines.

[l33tmeatwad]

I tried it on x64 Win10 as well. But I couldn't reproduce it due to lack of useful information. Post your full script and the mediainfo of your media file. What program did you use to open the script? Have you tried to open other media files and it's still the same?
I'll post some more information tomorrow when I'm at my computer again, but it was several MP4s containing x264 encoded video. The script was literally just LWLibavVideoSource and nothing else and works with the x64 plugin. It is not even generating the index file with the x86 version, just displays access violation in the AvsPmod preview window with no specific error codes.

[manolito]

Just like HolyWu I could not reproduce it on my machine... :confused:

Hardware:
Core i5 (two physical cores plus two hyperthreading cores)
8 GB RAM

Software:
Win7 64
AVS+ 3.40 32bit
Latest HolyWu LSMASH build from Nov 2019
Old StaxRip 32bit version as the host software
Current X264 32bit MABS build by Ligh

Source:
Concert Show by Toto
AVC video 1280x720 @25fps progressive in MP4 container

Target format:
AVC video @704x396 @25fps progressive in MKV container

AVS Script:
LWLibavVideoSource("D:\Toto.mp4", cachefile="D:\Toto temp files\Toto.lwi", format="YUV420P8")
ChangeFPS(25.00)
Crop(0,0, -Width % 4,-Height % 4)
ColorMatrix(source=0,dest=2)
RequestLinear(rlim=50, clim=50)
ConvertToYV12()
Spline36Resize(704,396)
avstp_set_threads(1)
Prefetch(4)


Two things to note about the script:
1. The ChangeFPS call right after the source filter has the only purpose to force linear access. I could have used RequestLinear instead. In MT mode this is necessary, otherwise the speed will drop to a snail's pace. (FFMS2 has the same behavior)
See here:
https://forum.doom9.org/showthread.php?p=1883754#post1883754

2. The RequestLinear call after ColorMatrix ist required to prevent crashes in MT mode. It is also necessary to force MT_SERIALIZED for ColorMatrix and RequestLinear if MT is used.
See here:
https://forum.doom9.org/showthread.php?p=1865279#post1865279
The relevant information is in the third paragraph of the post.


But with these settings the latest LWLibavVideoSource versions are absolutely stable on my machine...

[Taurus]

But with these settings the latest LWLibavVideoSource versions are absolutely stable on my machine...

Same here...
Using almost identical settings as manolito mentioned.
No crash, no nothing, pur fun :D

[l33tmeatwad]

After testing a few things I found out what I think is triggering it. Apparently files that contain cover art (iTunes) appear to crash the x86 version, a recontainered version will work fine. It's odd that it works on the x64 version but not the x86 version. The following machines were used to test with the same settings:

Machine 1
Processor: Core i7-8700 @ 3.20GHz
RAM: 16GB
Operating System: Windows 10 LTSB (2019) (64-bit)
Software: AviSynth+ 3.4.0
AvsPmod v2.6.1.1
Microsoft Visual C++ 2015-2019 Redist (14.24.28127)

Machine 2
Processor: Core i3-4160 @ 3.60GHz
RAM: 8GB
Operating System: Windows 10 LTSB (2019) (64-bit)
Software: AviSynth+ 3.4.0
AvsPmod v2.6.1.1
Microsoft Visual C++ 2015-2019 Redist (14.24.28127)

Machine 3
Processor: Core i5 @ 2.5GHz
RAM: 8GB
Operating System: macOS 10.11.6
Software: PlayOnMac (WINE, System Version 3.9)
AviSynth+ 3.4.0
AvsPmod v2.6.1.1

Media File
General
Complete name : 8bit Sample.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 60.5 MiB
Duration : 3 min 13 s
Overall bit rate : 2 616 kb/s
Cover : Yes

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3 min 13 s
Bit rate : 2 440 kb/s
Width : 960 pixels
Height : 720 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.147
Stream size : 56.4 MiB (93%)
Writing library : x264 core 133 r2334 a3ac64b
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 /
psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 /
trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 /
chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 /
nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 /
bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 /
open_gop=0 / weightp=2 / keyint=240 / keyint_min=23 / scenecut=40 /
intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.60 /
qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=17500 / vbv_bufsize=17500 /
crf_max=0.0 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 3 min 13 s
Duration_LastFrame : -19 ms
Bit rate mode : Constant
Bit rate : 169 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 3.90 MiB (6%)
Default : Yes
Alternate group : 1

AviSynth Script
LWLibavVideoSource("8bit Sample.mp4", format="YUV420P8")

[Taurus]

@l33tmeatwad:
I can confirm your findings.
Any other: directshowsource, dss2, ffvideoSource, etc. is doing fine.
This is on Win7_64bit with avisynth_32bit.
Have'nt tested avisynth_64bit.
Thanks for sharing your insight...

[l33tmeatwad]

Awesome, thanks for the speedy fix!

[Taurus]

:thanks:Thank you!

[outhud]

I want to compile L-SMASH-Works for Vapoursynth on Linux. Which repository should I use for this? Are there compile instructions somewhere?

[Are_]

https://github.com/HolyWu/L-SMASH-Works/

git clone https://github.com/HolyWu/L-SMASH-Works
cd L-SMASH-Works/VapourSynth/
mkdir build
meson . build/
ninja -C build/


https://github.com/l-smash/l-smash is a dependency for the vapoursynth plugin, besides ffmpeg and vapoursynth itself.

[outhud]

https://github.com/HolyWu/L-SMASH-Works/

git clone https://github.com/HolyWu/L-SMASH-Works
cd L-SMASH-Works/VapourSynth/
mkdir build
meson . build/
ninja -C build/


https://github.com/l-smash/l-smash is a dependency for the vapoursynth plugin, besides ffmpeg and vapoursynth itself.

Thanks. I've installed ffmpeg, vapoursynth and l-smash. It seems ninja fails on the last step for L-SMASH-Works with:

/usr/bin/ld: /usr/local/lib/liblsmash.a(dts.o): relocation R_X86_64_PC32 against symbol
`lsmash_remove_dts_reserved_box' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

[Are_]

Try to recompile l-smash with
./configure --disable-static --enable-shared

[outhud]

Try to recompile l-smash with
./configure --disable-static --enable-shared

That did it! Thanks.

[poisondeathray]

Lsmash seek /refresh issue with CPU decoding

If you seek to frame 240, or 480, or 720 then push refresh, the frame jumps to frame 0

This was posted from another forum;
https://www.mediafire.com/file/pvojgkqxu4t8mii/20151224_145719.m2ts/file


-Affects avspmod refresh, or vdub2 refresh, or vsedit refresh
-Affects x86/x64 , threads=1, avs/vpy versions, several different lsmash versions

-Decodes ok with ffms2

-Decodes ok with Lsmash decoder="h264_cuvid" (!) . But "wrong" framerate @ 59.94 . Correct frames without refresh bug. If you AssumeFPS(30000,1001), or repeat=true with selecteven() or selectodd(), it works work

-Only occurs on certain P-frames ; eg. if you go to frame 44 and refresh, it does not occur


Could it be related to frame doubling flag ? This from a Panasonic ZS7 / TZ10 that shoots 29.97p with frame doubling

Frame mode : Frame doubling

Problem still persists with repeat=true and selecteven() or selectodd()


Something to do with timecodes ? There is a discrepancy between VFR/CFR time reported by ffms2 ffinfo

[manolito]

[URL]
LWLibav: Change the default of 'repeat' to true.


Thanks a lot for the update...

Still I think that changing the default for the "repeat" parameter from false to true is not such a good idea. I do not dispute that setting it to true makes sense, my concern is that this change breaks almost all of my LSMASH based scripts.

So far the source filter behavior for all my filters has been like this:

1. MPEG2Source (and all DG source filters) honor the pulldown flags by default.

2. FFMS2 and LSMASH do not honor the pulldown flags by default.

3. DirectShow based source filters also do not honor the pulldown flags, but without specifying the fps parameter they output 29.97 fps, but not by honoring the pulldown flags, but by inserting duplicate frames instead.


I would really appreciate if all the source filter authors could agree on using the same method for treating pulldown flags - once and for all...


Cheers
manolito

[manolito]

Yes, I know that...

But:
This only applies if I do not do a VFR to CFR conversion by specifying fpsnum and fpsden (at least this is what the readme says). And since all my target formats require CFR I ALWAYS try to specify fpsnum and fpsden, even if the source already is CFR. If this is not possible because the host application does not expose the values for fpsnum and fpsden, I use ChangeFPS(23.976) right after the source filter call to do the VFR to CFR conversion.

And this is where the new default behavior breaks my scripts. Up to now with a soft telecined source LSMASH would spit out 23.976 fps, and the ChangeFPS command would make sure that I get perfect A/V sync (maybe at the cost of some duplicate or dropped frames). The new default behavior spits out 29.97 fps interlaced, and the following ChangeFPS(23.976) will probably ruin the output (because now IVTC is required).

[videoh]

Seems to me you need to fix your scripts. You can make a simple batch file to do it, yes? While you are at it, define everything and do not rely on defaults. Not meaning to be contrary, just pragmatic and realistic. I think it's a no-brainer to honor pulldown by default.

[manolito]

So far your source filters have been the only ones which do it that way. Everybody else (ffms2, LSMASH and DirectShowSource based filters) did it differently, and I kind of got used to it. That does not mean that I do not see the merits or your approach, but over the years I have established workflows which are mainly geared at other source filters than yours.

And at my age you do not change your old ways easily. Especially since I use ffms2 much more often than LSMASH, and so far ffms2 does not honor pulldown flags by default.

[videoh]

So now there are two important source filter authors that think the intent of the stream should be honored. There are many cases where irregular pulldown, hybrid content, etc., preclude an automatic rate conversion, such as you suggest, as it would destroy AV sync.

Keep the inner child alive. Like Sherman!

[manolito]

There are many cases where irregular pulldown, hybrid content, etc., preclude an automatic rate conversion, such as you suggest, as it would destroy AV sync.


Since I live in PAL land, I have never encountered such clips with irregular pulldown or hybrid content. Can anybody provide a link to such a clip?

[kedautinh12]

https://github.com/HolyWu/L-SMASH-Works/releases/download/20200118/L-SMASH-Works_20200118.7z


VideoSource: Fix missing frames in some H264 streams.
LWLibav: Change the default of 'repeat' to true.
LWLibav: Fix VP8 decoding issue with alt-ref frames.


Can you update support vfr to cfr like ffms2???

[manolito]

Can you update support vfr to cfr like ffms2???

I do not understand your problem... :confused:

ffms2 and LSMASH both support VFR to CFR conversion, and they use identical methods. As soon as the user specifies FPSNUM and FPSDEN in the parameter list, both tools will spit out a CFR clip. They do this by duplicating or dropping frames as necessary to reach the specified frame rate.

So what exactly is your problem with LSMASH?

[kedautinh12]

I do not understand your problem... :confused:

ffms2 and LSMASH both support VFR to CFR conversion, and they use identical methods. As soon as the user specifies FPSNUM and FPSDEN in the parameter list, both tools will spit out a CFR clip. They do this by duplicating or dropping frames as necessary to reach the specified frame rate.

So what exactly is your problem with LSMASH?

I changed VFR to CFR with FPSNUM and FPSDEN but video after encoded video don't sync with audio. But it's work with directshowsource mod

my script:
LSMASH: LWLibavVideoSource("C:\Users\84945\Downloads\Otome Shinto - Otome Shinto no Uta [1440x1080 h264 M-ON! HD].ts", fpsnum=30000, fpsden=1001)

and directshowsource mod: dss2("C:\Users\84945\Downloads\Otome Shinto - Otome Shinto no Uta [1440x1080 h264 M-ON! HD].ts", fps=29.970).AssumeFPS(30000,1001)
ConvertBits(8)

video test: https://drive.google.com/open?id=1wUKzCzij8HJl22ElhD8xMPkrUHv9p3OT

[sneaker_ger]

my script:
LSMASH: LWLibavVideoSource("C:\Users\84945\Downloads\Otome Shinto - Otome Shinto no Uta [1440x1080 h264 M-ON! HD].ts", fpsnum=30000, fpsden=1001)
Where is the audio in your script?

[kedautinh12]

Where is the audio in your script?

I use megui and use auto encode so don't need audio script

[sneaker_ger]

Sync is relation video to audio. You only provide us with info about video. How are we supposed to reproduce your test?

(Actually for me your sample is only a black screen with the "M O N" logo at the bottom right, just a small blink of the real video at the start.)

[kedautinh12]

Sync is relation video to audio. You only provide us with info about video. How are we supposed to reproduce your test?

(Actually for me your sample is only a black screen with the "M O N" logo at the bottom right, just a small blink of the real video at the start.)

I fixed with trim from frame 31 to end and cut audio like this with function avs cutter of megui but error vfr to cfr. When i used with directshowsource mod don't need trim and vfr to cfr work fine

[manolito]

I played with the uploaded sample a little bit. Just playing it in DirectShow based players works fine, but after converting it I get the same problem like sneaker_ger (just a small blink of the video at the start). So your sample seems to have problems to begin with...

Another likely cause for issues is the format. Interlaced HD transport streams are always a challenge for source filters, so I made it a habit to repack such sources into an MKV container before trying to convert them.

But the real problem about audio sync does come from the source filter. Captured transport streams often have a huge audio delay. Your sample has an audio delay of -632 ms, after repacking to MKV the delay is -631 ms (almost identical). And different source filters treat such delays differently.

For DirectShow based souce filters as well as for ffms2 my experience is that this delay has to be discarded when muxing converted audio and video streams. LSMASH hehaves differently, the delay needs to be honored for perfect audio sync. (I believe that the DG source filters behave identically). I think that the reason is that some source filters just delete video frames at the start before a valid key frame while others like LSMASH repeat the first valid video frame up to the point where the audio starts.

On my tests I got perfect audio sync with the uploaded sample by discarding the delay value under DSS2Mod and honoring the delay value for LSMASH.

[kedautinh12]

I played with the uploaded sample a little bit. Just playing it in DirectShow based players works fine, but after converting it I get the same problem like sneaker_ger (just a small blink of the video at the start). So your sample seems to have problems to begin with...

Another likely cause for issues is the format. Interlaced HD transport streams are always a challenge for source filters, so I made it a habit to repack such sources into an MKV container before trying to convert them.

But the real problem about audio sync does come from the source filter. Captured transport streams often have a huge audio delay. Your sample has an audio delay of -632 ms, after repacking to MKV the delay is -631 ms (almost identical). And different source filters treat such delays differently.

For DirectShow based souce filters as well as for ffms2 my experience is that this delay has to be discarded when muxing converted audio and video streams. LSMASH hehaves differently, the delay needs to be honored for perfect audio sync. (I believe that the DG source filters behave identically). I think that the reason is that some source filters just delete video frames at the start before a valid key frame while others like LSMASH repeat the first valid video frame up to the point where the audio starts.

On my tests I got perfect audio sync with the uploaded sample by discarding the delay value under DSS2Mod and honoring the delay value for LSMASH.

I think error don't relate with delay of audio, i used aegisub and make karaoke sub sync audio and video and after encoded only audio sync with karaoke sub but video

[kedautinh12]

I am wondering how did you determine the sync issue was caused by VFR->CFR conversion. Have you tried omitting the fpsnum and fpsden arguments? And was there even VFR->CFR conversion taking place? It seems a no-op to me since the frame rate of your sample is already 29.97 (30000/1001) fps.

Why i use vfr to cfr same fps in directshowsource mod (value fps in script of directshowsource mod are autoset by megui) and it's work perfectly??

[manolito]

Since I live in PAL land, I have never encountered such clips with irregular pulldown or hybrid content. Can anybody provide a link to such a clip?

Bump...

[hello_hello]

Does the repeat option change the way the video is decoded in respect to chroma subsampling? I'm just wondering what happens when it's set to true and a source is progressive? I've never really understood how chroma subsampling is decoded in that respect.

repeat
Reconstruct frames by the flags specified in video stream and then treat all frames as interlaced if set to true and usable.

Is it safe to assume the ability to use index files as the source will remain broken?

[hello_hello]

So now there are two important source filter authors that think the intent of the stream should be honored. There are many cases where irregular pulldown, hybrid content, etc., preclude an automatic rate conversion, such as you suggest, as it would destroy AV sync.

Would it destroy the AV sync? The decoder just drops or duplicates frames for a constant frame rate.

If you have a DVD that's a mixture of soft telecined 23.976fps and 29.97fps progressive or interlaced, and repeat is false but you use FPSNum and FPSDen to convert to a constant frame rate of 29.97fps, you should end up with the progressive sections having every 4th frame repeated and the audio sync should be intact, unless I'm missing something. I'm not saying it's ideal, but unless you want a VFR output, it needs to be one or the other. It's only if you don't enable repeat flags or do a VFR conversion that you'll end up with audio sync problems.

The help file says the repeat option is ignored when VFR to CFR conversion is specified, which might be why it's always been false by default, although I'm not sure how often you'd want a CFR to VFR conversion and honour repeat flags at the same time.

ffms2 can write a timecodes file as it indexes, so in that respect, not honouring repeat flags by default probably makes sense as you can use them for a VFR encode. There'd be no need to honour repeat flags and then use TIVTC to run an analysis pass to return the frame rate to variable.

I don't recall lsmash having an option to write a timecodes file though, so repeat being true by default probably makes sense.

[videoh]

Bump... Here is one:

http://rationalqm.us/misc/lainvob.vob

It's a small sample provided under fair use. I have some larger off-air TS captures that are similar. This stuff is ubiquitous in NTSC land.

[videoh]

It's only if you don't enable repeat flags ... that you'll end up with audio sync problems. Correct, and that's the point because manolito wants to ignore pulldown by default (he objects to honoring pulldown by default because it breaks his existing scripts).

[manolito]

Here is one:

http://rationalqm.us/misc/lainvob.vob

It's a small sample provided under fair use. I have some larger off-air TS captures that are similar. This stuff is ubiquitous in NTSC land.

Thanks for the clip, but this one does not do anything for me...

I know that you are a "pedal to the metal" guy, and I prefer encoder GUIs.

First of all the sample clip is not suitable to judge A/V sync. And then most encoder GUIs use MediaInfo to determine the source properties, and this clip appears as 29.97 interlaced TFF. No pulldown detected. And the encoder treats the clip exactly like this.

I used AVStoDVD to compare conversions first using DGIndex and then using ffms2. Both looked identical to me.

To get meaningful results the source clip needs some scenes where I can judge A/V sync, and then it would help if MediaInfo would detect 3:2 Pulldown. If MediaInfo just detects 29.97 pure interlaced then there is no way that the encoder GUI could do any fancy things.


So it all comes down to the philosophy you apply. I want things to work universally under an encoder GUI like AVStoDVD or StaxRip. You are able to do all this stuff manually and find the best way to encode such clips, but this is something most users cannot (and do not want to) do.

[videoh]

The clip has irregular pulldown (which is what you asked for). If you want a longer sample, buy the DVD.

I want things to work universally... I want a unicorn for Christmas. ;)

And, hey, I'm not using any super powers here. A quick preview in DGIndex clearly shows the irregular pulldown. Press F5 and let it play to the end. Then look at the field repeats in the info dialog and the film percentage in the DGI file.

If you have a clip with hybrid soft and hard pulldown, you can only honor pulldown and then do IVTC with a filter. Ignoring the pulldown would ruin AV sync. All of this is well known in NTSC land.

[manolito]

A quick preview in DGIndex clearly shows the irregular pulldown. Press F5 and let it play to the end. Then look at the repeat fields and film percent in the info dialog.


No "Average Joe" user would ever want to do this. You are from Pluto and I am from Mars...

[kedautinh12]

Repeat
Reconstruct frames by the flags specified in video stream and then treat all frames as interlaced if set to true and usable.

I test many raw with repeat and still interlaced in some frame, i think it's only use with raw have duration of video don't sync with duration of audio

[videoh]

No "Average Joe" user would ever want to do this. You are from Pluto and I am from Mars... Average Joes also do stuff like deinterlace 3:2 pulldown, and resize before deinterlacing. This is Doom9, where things are done right! Maybe VideoHelp is more your style?

[videoh]

I test many raw with repeat and still interlaced in some frame, i think it's only use with raw have duration of video don't sync with duration of audio What do you mean by "raw"? Of course, 3:2 pulldown is going to appear to have interlaced frames. So I don't get your point.

[kedautinh12]

What do you mean by "raw"? Of course, 3:2 pulldown is going to appear to have interlaced frames. So I don't get your point.

Oh, sorry i call .ts file is raw, my mistake

[manolito]

This is Doom9, where things are done right! Maybe VideoHelp is more your style?

Being your old arrogant self again...:p

[videoh]

Call it confidence born of deep theoretical understanding and a broad base of experience.

[hello_hello]

The clip seems to be a mixture of soft and hard telecine and interlaced/progressive. I tried 4 different decoding/filtering methods, all using ffms2 as I wanted to try a sample using the timecodes file it (optionally) creates when indexing.
RFFMode=0 is the same as Repeat=false for lsmash.

1 VFR ffms2 timecodes - 1762 frames.mkv
The hard telecine parts are field matched and left at 29.97fps. Average frame rate 27.583fps.
FFVideoSource("E:\lainvob.mkv", Threads=1, TimeCodes="FFMSTimecodes.txt", RFFMode=0)
TFM()

2 VFR TIVTC 2 pass - 1662 frames.mkv
Same as #1 except the hard telecine parts are decimated to 23.976fps. Average frame rate 26.028fps.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=1)
TFM(Input="D:\TFM.txt")
TDecimate(Mode=5, Hybrid=2, Input="D:\TDec.txt", tfmin="D:\TFM.txt", mkvout="D:\Times.txt")

3 CFR conversion ffms2 - 1916 frames.mkv
The hard telecine parts are field matched and left at 29.97fps. The soft telecine parts would have every 4th frame duplicated.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=0, FPSNum=30000, FPSDen=1001)
TFM()

4 CFR conversion TIVTC frame blending - 1916 frames.mkv
The hard telecine parts are field matched and decimated, and the soft/hard telecined parts are frame blended to 29.97fps.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=1)
TFM()
TDecimate(Mode=1, Hybrid=3)

All cropped and resized to 640x480.
encodes.zip (https://www.sendspace.com/file/mriedq) (72.6 MB)

And one more at the average frame rate of 27.568fps (1762 frames). The A/V sync is definitely different. At roughly the 41 second point she should be mouthing the word "much" in sync with the audio, as happens with all the previous samples.
FFVideoSource("E:\lainvob.mkv", Threads=1, RFFMode=0)
TFM()
Average frame rate.mkv (https://www.sendspace.com/file/fqrl6q)

manolito,
for a method that always works with encoder GUI's wouldn't it be better to always honour repeat flags as DGIndex does by default? I don't understand the logic behind not doing so unless you want a VFR output, or there's not much soft telecined content and the repeated frames wouldn't be too noticeable. Admittedly PAL is generally much simpler to deal with, but for NTSC you can always tell if there's soft pulldown by disabling repeat flags and using Info() to check the frame rate.

[hello_hello]

repeat
Reconstruct frames by the flags specified in video stream and then treat all frames as interlaced if set to true and usable.

I'm still not clear on whether the repeat flag effects the chroma upsampling (interlaced vs progressive) or am I looking at it all wrong?

[manolito]

First of all I really do not want to do a thorough analysis of such clips using DGIndex. And I absolutely want to avoid VFR, all my conversions need to come out as CFR.

The 2 GUIs I use are AVStoDVD for DVD output and StaxRip 32-bit for other output formats (I also use DVDStyler and dmMediaConverter, but they do not count here because they do no use AviSynth). And both GUIs employ MediaInfo to detect the source properties.

The clip which was uploaded by our friend is detected as 29.97 fps Interlaced TFF. No soft pulldown is detected. MediaInfo only reports 2:3 pulldown if the clip has the standard regular pulldown pattern for 23.976 to 29.97 telecining.

Both GUIs ALWAYS do VFR to CFR conversion right at the source filter level. If the source aready is CFR then this operation is useless, but it won't do any harm either.

The uploaded clip will be treated like this:
DGIndex / DGDecode:
The default "Honor Pulldown Flags" is used, followed by a ChangeFPS(29.97). Then users can either leave it interlaced (the default) or deinterlace the clip. IVTC is not offered because MediaInfo did not report pulldown.

DSS2Mod (the same for DirectShowSource):
DSS2Mod does not honor pulldown flags. The uploaded clip will have the "fps=29.97" parameter in the DSS2Mod call, so there will be lots of dupes. Then again you can leave it interlaced or use a deinterlacer.

ffms2 and previous LSMASH builds:
Pretty much the same as DSS2Mod.

This is not optimal, but it is very watchable, and I never saw any A/V sync problems.


If the source has regular 2:3 pulldown then it gets a little different. MediaInfo will report 23.976 progressive and 2:3 pulldown. When using DGIndex with the default Honor Pulldown Flags then AVStoDVD will force an IVTC right after the source filter. When DSS2Mod (or ffms2 or previous LSMASH) is used then the source filter will force VFR to CFR conversion to 23.976 fps. There will be dropped or duplicated frames, but again no A/V sync problems, and the output will be pure progressive.

[manolito]

for a method that always works with encoder GUI's wouldn't it be better to always honour repeat flags as DGIndex does by default?

Main reason is that DSS2Mod is my standard source filter (AVSToDVD uses it by default), and all DirectShow based source filters cannot honor pulldown flags.

I found this out the hard way because it is not documented anywhere.

If your source is a regular soft telecined NTSC Film clip and you open it with DSS2Mod (or DirectShowSource) without the "fps=" parameter then the output will be 29.97 fps. For some time I thought that DSS2Mod did indeed honor the pulldown flags, but I was quite wrong. The output had no pulldown pattern, it was purely progressive with lots of dupes. Which means that DSS2Mod decoded the clip to 23.976 first and then added dupes to bring the rate up to 29.97.

I still think that this is a stupid behavior, so for sources with 2:3 pulldown I always specify "fps=23.976" in the DSS2Mod parameter list.

[hello_hello]

Both GUIs ALWAYS do VFR to CFR conversion right at the source filter level. If the source aready is CFR then this operation is useless, but it won't do any harm either.

It can. I haven't tested with vob files, but I've seen the problem with both ffms2 and lsmash for AVI and MKV sources. I think it's something to do with jitter in the source timecodes, but it's possible that when the frame rate is constant and you tell ffms2 or lsmash to convert to that constant frame rate, they'll drop and repeat frames in places when in theory they should do nothing. As a result I never use frame rate conversion unless I want to convert the frame rate.
https://forum.doom9.org/showpost.php?p=1703007&postcount=4395

Anyway... there's definitely soft pulldown in that clip, unless I'm interpreting it wrong. Just not at the beginning, which is probably where MediaInfo looks.

If you remux the sample as an MKV and extract the timecodes, you'll see they're 33ms apart at the beginning, which means either progressive, interlaced, or hard pulldown, all at 29.97fps.

If you scroll down to frame number 652, the timecodes look like this:

22039
22089
22122
22172
22206
22256
22289
22339
22372
22422
22456
22506

The only explanation I can think of for the timecodes being 50ms apart is due to the repeat flags not being included in the timecodes.

1000 / (30000 / 1001) = 33.37 ms per frame.
1.5 frames = 50.05 ms

But does that mean if you don't honour pulldown flags, instead of telecined frames at 33.37 ms per frame, or progressive frames at 41.7ms per frame (23.976fps) after decimation, you're getting frames alternating between 50ms and 33ms for an average of 41.5ms (roughly)?
I hadn't thought about it like that until now. Mind if you do a VFR to CFR conversion with DSS or Repeat=true etc, it'd probably even them out to 33.37ms durations, with every forth frame repeated, or to an even 41.7ms per frame if you convert to 23.976fps.

[edcrfv94]

Vapoursynth R45 not support?