Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd September 2023, 18:27   #2761  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
ffms2_r1369 (pass: BLxmx6YsVJ03):
- AviSynth: fixed GetParity (rffmode > 0) and _FieldBased. (bugs introduced in the previous version)
- AviSynth: added support for audio channel mask.
- AviSynth: fixed fps (rffmode > 0). (bug introduced by the new FFmpeg API from the beginning of 2023)
- ffms2@cf7c4b2;
- ffmpeg@9310ff;
- zlib 1.3;
- dav1d 1.2.1;
- libxml2 v2.11.5.

Quote:
Originally Posted by rgr View Post
Is it a known issue that ffms2 doesn't respect "Delay relative to video : -200ms"?
Does ffmpeg respect it by default?
StvG is offline   Reply With Quote
Old 23rd September 2023, 10:15   #2762  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,790
Quote:
Originally Posted by StvG View Post
ffms2_r1369 (pass: BLxmx6YsVJ03):
...
- AviSynth: added support for audio channel mask.
...
Thanks, to obtain the audio channel mask with last avs+ we need before:

global OPT_UseWaveExtensible = true # Needed to use global OPT_dwChannelMask=
audio=FFAudioSource("8w3D.ec3")
global OPT_dwChannelMask=FFCHANNEL_LAYOUT

Now is enough:

audio=FFAudioSource("8w3D.ec3")

to obtain the channel mask if is included in the source.
__________________
BeHappy, AviSynth audio transcoder.
tebasuna51 is offline   Reply With Quote
Old 28th September 2023, 11:01   #2763  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,738
Quote:
Originally Posted by StvG View Post
- AviSynth: fixed GetParity (rffmode > 0) and _FieldBased. (bugs introduced in the previous version)
- AviSynth: added support for audio channel mask.
- AviSynth: fixed fps (rffmode > 0). (bug introduced by the new FFmpeg API from the beginning of 2023)
Thank you for fixing those and most importantly for supporting audio channel masks.
Tebasuna and I are probably the only two people excited for it, but still, thank you! XD
FranceBB is offline   Reply With Quote
Old 28th September 2023, 11:45   #2764  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,082
Me too
kedautinh12 is offline   Reply With Quote
Old 28th September 2023, 12:52   #2765  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,531
Quote:
Originally Posted by StvG View Post
ffms2_r1369 (pass: BLxmx6YsVJ03):
- AviSynth: fixed GetParity (rffmode > 0) and _FieldBased. (bugs introduced in the previous version)
- AviSynth: added support for audio channel mask.
- AviSynth: fixed fps (rffmode > 0). (bug introduced by the new FFmpeg API from the beginning of 2023)
- ffms2@cf7c4b2;
- ffmpeg@9310ff;
- zlib 1.3;
- dav1d 1.2.1;
- libxml2 v2.11.5.

Does ffmpeg respect it by default?
Is the current patch set available somewhere?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 29th September 2023, 09:40   #2766  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,790
Quote:
Originally Posted by Myrsloik View Post
Is the current patch set available somewhere?
It is included in the download.

Of course you need also the last avisynth.h
__________________
BeHappy, AviSynth audio transcoder.

Last edited by tebasuna51; 29th September 2023 at 09:55. Reason: add info
tebasuna51 is offline   Reply With Quote
Old 29th September 2023, 09:53   #2767  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,531
Quote:
Originally Posted by tebasuna51 View Post
It is included in the download.
Oh, only as a single monolithic mess with no comments as to why certain parts are relevant? That's a lot less helpful.

I did however see that getparity is implemented for avisynth. Does anything actually use that?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 29th September 2023, 09:59   #2768  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,082
Quote:
Originally Posted by Myrsloik View Post
Oh, only as a single monolithic mess with no comments as to why certain parts are relevant? That's a lot less helpful.

I did however see that getparity is implemented for avisynth. Does anything actually use that?
I think avisynth more helpful than vapoursynth thread of doom9
kedautinh12 is offline   Reply With Quote
Old 22nd October 2023, 03:28   #2769  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
Just a heads up that from this version there is a regression about the handling of the unicode filenames.

If you have issues about unicode filenames complain to the used app for reading/opening the avs script that doesn't support unicode filenames or just active the Unicode UTF-8 support in Windows (if available).

Starting from this post you can read about AvsPmod.

I'll upload the code used for my builds to codeberg and then post a new version.
StvG is offline   Reply With Quote
Old 22nd October 2023, 09:13   #2770  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
Seeking test with ffms2_r1369 and ffms2 build from f20827c (both using the same versions of the external libraries). The used video files.

StvG is offline   Reply With Quote
Old 22nd October 2023, 09:52   #2771  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,738
In other words, from the chart and the test assets, there's no scenario in which someone should be using the legacy Myrsloik 2020 version given that the new version 2023 builds you keep producing addressed plenty of the issues it had.
Once again, thank you for keeping the project alive, in fact your builds have now become the standard included in FFAStrans and also the one my colleagues and I use at work on a daily basis
FranceBB is offline   Reply With Quote
Old 22nd October 2023, 10:23   #2772  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
Also you probably noticed that there are files with same names but different containers. I remuxed the files from ts,mts,mpg,mp4... to mkv. Other than VC-1 the other codecs are ok when mkv is used.
StvG is offline   Reply With Quote
Old 4th November 2023, 20:51   #2773  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,738
Ok, so, after some rather extensive testing and after wondering "why?" many times, I actually realized one thing: FFMpeg's mxf muxer for MJPEG2000 RGB48 HDR PQ files is now completely broken.
It already had its own issues a year ago with the wrong bit depth being written in the container and the CDCI essence descriptor UL (i.e YUV 4:4:4) being used instead of the RGBA essence descriptor UL (i.e RGB48), but now it got worse.
You now may be wondering: "Well, that's an ffmpeg problem, what does it have to do with FFVideoSource()"?
Well, simply enough, now the video in the mxf container remuxed by ffmpeg is so broken that FFVideoSource() - albeit indexing it - shows repeated/duplicated frames all over the place while dropping some other frames (I tested both ffms2 version r1347 from January 2023 and version r1369 from September 2023) and they both have issues. I even tried different seeking modes like forcing linear, specifying fpsnum=24000 and fpsden=1001 etc but nothing really helped.
Please note that this is not necessarily a problem of ffms2, but rather FFMpeg muxing the file blatantly incorrectly in the mxf container.
On the other hand, muxing the very same file in the mkv container has no issues at all and in fact both versions of ffms2 index it just fine and without any repeated/duplicated frames, correctly reporting 23,976p.

So, this post is to warn everyone and avoid having other people spending hours in debugging:
if anyone has to deal with MJPEG2000 files in mxf muxed by FFMpeg, please remux them in mkv and they'll work like a charm.


For reference, here's the original mxf file:

Quote:
General
Complete name : Test.mxf
File size : 295 GiB
Duration : 57 min 30 s
Overall bit rate : 734 Mb/s
Frame rate : 23.976 FPS

Video
ID : 2-2
Format : JPEG 2000
Format profile : IMFS4k@ML6SL4
HDR format : SMPTE ST 2086, HDR10 compatible
Muxing mode : MXF
Codec ID : 0D010301020C0600-0401020203010313
Duration : 57 min 30 s
Bit rate : 716 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : RGB
Bit depth : 12 bits
Scan type : Progressive

Bits/(Pixel*Frame) : 3.600
Stream size : 288 GiB (97%)
Title : Image Track
Color range : Full
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
and here's what happens after remuxing it with ffmpeg's mxf muxer:


Quote:
General
Complete name : Test.mxf
Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 295 GiB
Duration : 57 min 30 s
Overall bit rate : 734 Mb/s
Frame rate : 23.976 FPS
Package name : File Package: SMPTE ST 422 / ST 2067-5 frame wrapping of JPEG 2000 codestreams
Writing application : FFmpeg OP1a Muxer
Writing library : Lavf (mingw32)

Video
ID : 2
Format : JPEG 2000
Format profile : IMFS4k@ML6SL4
HDR format : SMPTE ST 2086, HDR10 compatible
Format settings, wrapping mode : Frame
Codec ID : 0D010301020C0100-0401020203010100
Duration : 57 min 30 s
Bit rate : 716 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 16 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 3.600
Stream size : 288 GiB (97%)
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Delay_SDTI : 3573570

In other words the container is lying as it's saying that it's a 4:4:4 16bit file, while it's actually an RGB48 file, which causes FFVideoSource() to be rather "confused" about it.
The range information (originally full range) is also lost, which of course is very bad for any automation you might have depending on it to be set correctly in the Avisynth frame properties.
On the other hand, remuxing it to .mkv has no issues at all and preserves everything correctly.
So, in a nutshell, if you get an FFMpeg muxed MJPEG2000 in mxf, please remux it to a sensible container like mkv before indexing it, otherwise the result is gonna be completely wrong.

Last edited by FranceBB; 6th November 2023 at 09:07.
FranceBB is offline   Reply With Quote
Old 10th November 2023, 15:25   #2774  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,738
Quote:
Originally Posted by StvG View Post
ffms2_r1369 (pass: BLxmx6YsVJ03):
- AviSynth: fixed GetParity (rffmode > 0) and _FieldBased. (bugs introduced in the previous version)
- AviSynth: added support for audio channel mask.
- AviSynth: fixed fps (rffmode > 0). (bug introduced by the new FFmpeg API from the beginning of 2023)
- ffms2@cf7c4b2;
- ffmpeg@9310ff;
- zlib 1.3;
- dav1d 1.2.1;
- libxml2 v2.11.5.
Don't hate me, but I'm still having issues with files that have Italian accents.
FranceBB is offline   Reply With Quote
Old 11th November 2023, 11:49   #2775  |  Link
LeXXuz
21 years and counting...
 
LeXXuz's Avatar
 
Join Date: Oct 2002
Location: Germany
Posts: 716
Quote:
Originally Posted by FranceBB View Post
Don't hate me, but I'm still having issues with files that have Italian accents.
Same here with german characters. I stopped using ffms and LSmash some time ago. At least DGIndex doesn't have the slightest problem with special characters in file names. And it is still the most precise indexer imho as long as you use it on elementary steams and not containers.
LeXXuz is offline   Reply With Quote
Old 11th November 2023, 15:51   #2776  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
Quote:
Originally Posted by FranceBB View Post
Don't hate me, but I'm still having issues with files that have Italian accents.
Quote:
Originally Posted by LeXXuz View Post
Same here with german characters. I stopped using ffms and LSmash some time ago. At least DGIndex doesn't have the slightest problem with special characters in file names. And it is still the most precise indexer imho as long as you use it on elementary steams and not containers.
Post #2769.
StvG is offline   Reply With Quote
Old 11th November 2023, 19:39   #2777  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,738
Quote:
Originally Posted by StvG View Post
If you have issues about unicode filenames complain to the used app for reading/opening the avs script that doesn't support unicode filenames
That is quite literally FFMpeg, which, in theory, does, so there must be something else going on here.
Validating scripts with ffprobe and ffmpeg results in an error on both sides.
I'll come up with more examples on Monday.
FranceBB is offline   Reply With Quote
Old 11th November 2023, 19:53   #2778  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
Quote:
Originally Posted by StvG View Post
Just a heads up that from this version there is a regression about the handling of the unicode filenames...

... I'll upload the code used for my builds to codeberg and then post a new version.
Wait for the new version or use old version.
StvG is offline   Reply With Quote
Old 12th November 2023, 01:32   #2779  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 422
The source code is here.

ffms2_r1386

@FranceBB, you can try this version.

If AvsPmod is used, make sure the used version is >=2.7.5.5 to avoid issues with the file names.
StvG is offline   Reply With Quote
Old 12th November 2023, 02:32   #2780  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,738
Quote:
Originally Posted by StvG View Post
The source code is here.

ffms2_r1386

@FranceBB, you can try this version.
Woah, thank you a lot, gotta try it straight away!

Ok, so, tested on a real life workflow (slowmotion from 100fps to 25fps and encode in XDCAM-50) with a simple file called:

Test accenti .MP4


Version ffms2_r1369 (old):
Validate: FFVideoSource: Failed to open 'Test accenti *èéùì.MP4'




Version ffms2_r1386 (new):

the encoding started just fine



and went on to complete just fine.



Thank you, thank you, thank you!
I upgraded all my production systems.
This is gonna be a very nice surprise for the news department guys as they clock in just a few hours, at 5AM.
FranceBB is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:05.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.