Testing AviSynth+ frame properties in FFmpeg/mpv

[MysteryX]

so ... in VapourSynth we cheat and read properties from the first frame for simplicity. In Avisynth, any usage of frame properties must be in ScriptClip?

When you need to pass ColorRange to create a LUT table, or other complex filters, it's kind of a dead-end. Creating a LUT on every frame killed performance. For something simple like calling Tweak with the right coring parameter.

[StvG]

Originally Posted by MysteryX

so ... in VapourSynth we cheat and read properties from the first frame for simplicity. In Avisynth, any usage of frame properties must be in ScriptClip?

When you need to pass ColorRange to create a LUT table, or other complex filters, it's kind of a dead-end. Creating a LUT on every frame killed performance. For something simple like calling Tweak with the right coring parameter.

Read frame prop from the first frame, save it to temp file, read the file. An example (RT_Stats, CallCmd used) :

BlankClip()
propset("_reba", 4)
temp_dir = RT_GetSystemEnv("temp") + "\" + "temp.txt"

ScriptClip(function [temp_dir] ()
            {
               RT_TxtWriteFile(String(propGetInt("_reba")), temp_dir)
               last
            })
        RT_AverageLuma(n=0, w=1, h=1)

var = Value(RT_ReadTxtFromFile(temp_dir, 1, 0))
Subtitle(String(var))
ScriptClip(function [temp_dir] () { CallCmd(close="cmd /c del " + temp_dir, hide=true)  })

[StainlessS]

Nuttin' stopping you from something like this [the cheating stuff]

AviSource(...)
current_frame = 0
x=Averageluma # or whatever frame property thingy
...

[StainlessS]

This works

BlankClip()
propset("_reba", 4)

# ...

# cheatin' stuff

current_frame=0  # pretend frameserving in runtime filter
var = propGetInt("_reba")
Subtitle(String(var))    # EDIT: And then drop-though to frame serve to Player, Vdub etc

[MysteryX]

I had written the VapourSynth code to respect per-frame color-range and matrix -- in the theoretical case that it could ever be useful.

But I came to the conclusion that in such rare case, you'd be better off running a filter that auto-splits the file into segments to process them individually, because nearly no filter would support respecting it per-frame anyway.

And for theoretical or imaginary benefits, you pay a performance cost, and cannot use LUT or any hard-to-initialize filter based on frame property values.

Is there any Input (video source) or Output (encoder/player) that supports variable matrix and range?

[FranceBB]

Today I had a very bad behavior with frame properties.
A source encoded by a broadcast hardware recording port passed some dynamically changing metadata; in other words, they have been constant up to a certain point and then they changed 'cause the input stream into the port changed and the saved metadata in the file changed accordingly.
When I indexed it with ffms3 and worked with the file inside Avisynth with several filters like dfttest etc it went through up to a certain point, then the encoding just stopped.
It didn't crash, the RAM was still allocated, but the encoding just stopped, without any sign of moving/proceeding.
I repeated the test over and over again, it was always stopping at the very same point.
This happened with FFMpeg opening the Avisynth Script and encoding.
I tried to re-encode the source in HuffYUV lossless with FFMpeg and then Index with ffms3 and encode, same behavior, stopped mid encode.
I tried to re-encode the source in H.264 lossless with FFMpeg and then Index with ffms3 and encode, same behavior, stopped mid encode.
I tried to re-encode the source in FFV1 lossless with FFMpeg and then Index with ffms3 and encode, same behavior, stopped mid encode.
Then, I realized it could have been because of the dynamically changing metadata, so I re-encoded with FFMpeg in FFV1 lossless but stripping out the metadata like so:

-c:v ffv1  -slicecrc 1 -c:a pcm_s24le -ac %i_a_channels% -pix_fmt yuv422p10le -map_metadata -1 -s %i_width%:%i_height% -ar 48000

and then indexed with ffms3 and encoded and... it worked like a charm.
So the question is: is there a way to make ffms3 ignore and strip out the metadata / frame properties entirely to prevent this kind of thing from happening and behave like the old good ffms2 + Avisynth 3.7.0?

[FranceBB]

Nice one, StainlessS!
I'm reading about http://avisynth.nl/index.php/Internal_functions#propSet

Looks like it works!
So now the user is accountable to get and set the right frame properties for every encode every time.
What a bummer DX

Jokes aside, it works, this is an HDR PQ:

FFVideoSource("\\mibctvan000\Ingest\MEDIA\temp\trim.m2ts")

and this is after the conversion to HLG:

FFVideoSource("\\mibctvan000\Ingest\MEDIA\temp\trim.m2ts")


ConvertYUVtoLinearRGB(Color=0,OOTF=false)
ConvertLinearRGBtoYUV(Color=0,HDRMode=1,HLGLw=1200,HLGColor=2,OOTF=false)
propset("_Transfer", 18)
propDelete("ContentLightLevelAverage")
propDelete("MasteringDisplayMaxLuminance")
propDelete("MasteringDisplayMinLuminance")
propDelete("MasteringDisplayPrimariesX")
propDelete("MasteringDisplayPrimariesY")
propDelete("MasteringDisplayWhitePointX")
propDelete("MasteringDisplayWhitePointY")
propDelete("ContentLightLevelMax")

So what is cool about this is that we CAN not only make use of the properties like automatizing tonemapping by reading the MaxCLL value and set it as a variable, but we can also pass the right info to whatever we're frameserving to so that it's gonna make use of those info.

The "bummer" is that the user MUST set those correctly every time and has to do that manually...

or... just cheat and turn Avisynth 3.7.1 into Avisynth 3.7.0 with:

propClearAll()

hahahhahahahahahaha

[videoh]

I'm trying to implement the AVS+ frame properties for DGDecodeNV but getting confused. I took avisynth.h and the avs folder from pinterf's GIT. I have in GetFrame():

PVideoFrame frame = env->NewVideoFrame(vi);

Then I try to get props:

AVSMap* props = m_decoder.has_at_least_v8 ? env->getFramePropsRW(frame) : nullptr;

But there is an error because getFramePropsRW() wants an AVSFrameRef*. But I never heard of an AVSFrameRef.

Can someone please straighten me out on how to do this? Thank you.

[pinterf]

Originally Posted by videoh

I'm trying to implement the AVS+ frame properties for DGDecodeNV but getting confused. I took avisynth.h and the avs folder from pinterf's GIT. I have in GetFrame():

PVideoFrame frame = env->NewVideoFrame(vi);

Then I try to get props:

AVSMap* props = m_decoder.has_at_least_v8 ? env->getFramePropsRW(frame) : nullptr;

But there is an error because getFramePropsRW() wants an AVSFrameRef*. But I never heard of an AVSFrameRef.

Can someone please straighten me out on how to do this? Thank you.

Use the central AvisynthPlus repo, mine is not a fresh one.

[videoh]

Thank you, pinterf. I'll try that in the morning and report back.

There's one other matter I can work around but thought to mention. If I have a filter that supports VS and AVS, then I have to include both header files. But then they both define ptUnset etc. in enums, causing a compiler error.

[pinterf]

For this very reason the names were changed in current avs header.

[videoh]

Originally Posted by pinterf

Use the central AvisynthPlus repo, mine is not a fresh one.

That's working great. Thank you.

[Reel.Deel]

@qyot27

Would it be possible for AviSynth+ to pass a timecode file (VFR) to FFmpeg?

[tormento]

Originally Posted by Reel.Deel

Would it be possible for AviSynth+ to pass a timecode file (VFR) to FFmpeg?

If interested, I posted a message in the AVS main development thread and in MKVToolnix one too.

AFAIK, for now, it's easier to encode as if it is a CFR and then apply a timestamps file to MKV.

[Reel.Deel]

Originally Posted by tormento

AFAIK, for now, it's easier to encode as if it is a CFR and then apply a timestamps file to MKV.

I know how to insert timecodes into mp4/mkv files. My use case is different. Creating animated PNGs/GIFs/Webp with AviSynth/FFmpeg.

[qyot27]

Originally Posted by Reel.Deel

@qyot27

Would it be possible for AviSynth+ to pass a timecode file (VFR) to FFmpeg?

Theoretically, the AviSynth demuxer could be adjusted to use the given values of the frame timing (I assume that's the _AbsoluteTime, _DurationNum, and _DurationDen frame properties, or whatever can be used to modify them per-frame), and iterate it over every frame instead of doing what it currently does, which is just set the framerate of the entire clip using the num/den rational.

The logic for that would likely have to shift location significantly, probably out of avisynth_create_stream_video and into avisynth_read_packet_video, or constitute a hybrid where the current approach works in concert with whatever frame properties are set as overrides. It's largely the same reason that I didn't add the MasteringDisplay and whatnot properties - as those don't have to be static, making sure they're set right is trickier (also, AFAIK, FFmpeg doesn't set those as properties, but as side metadata, requiring a much larger amount of changes to even get the AviSynth demuxer to work with that concept to start with).

Can it be done? Yeah, probably. It's currently above my head, though (and I rarely ever have to deal with VFR as it is, so ensuring it does what it's supposed to do would be up to the community). If someone wants to figure it out and contribute, by all means, have at it.

[qyot27]

FFmpeg now supports reading two more frame properties - _SARNum and _SARDen, which are then combined into what FFmpeg reads as the full SAR value.

And because of how...tricky...that one would be when dealing with extraordinarily common resizing tasks, there is now a brand-new mechanism:

>ffmpeg -hide_banner -h demuxer=avisynth
Demuxer avisynth [AviSynth script]:
    Common extensions: avs.
AviSynth demuxer AVOptions:
  -avisynth_flags    <flags>      .D......... set flags related to reading frame properties from script (AviSynth+ v3.7.1 or higher) (default field_order+range+primaries+transfer+matrix+chroma_location)
     field_order                  .D......... read field order
     range                        .D......... read color range
     primaries                    .D......... read color primaries
     transfer                     .D......... read color transfer characteristics
     matrix                       .D......... read matrix coefficients
     chroma_location              .D......... read chroma location
     sar                          .D......... read sample aspect ratio

So using the old LG 4K HDR New York demo,
>ffmpeg -hide_banner -loglevel 40 -i test_hdr.avs

Stream #0:0: Video: rawvideo, 1 reference frame (Y3[11][10] / 0xA0B3359), yuv420p10le(tv, bt2020nc/bt2020/smpte2084, progressive, left), 3840x2160, 25 fps, 25 tbr, 25 tbn

And with -avisynth_flags:
>ffmpeg -hide_banner -loglevel 40 -avisynth_flags +sar-field_order-chroma_location -i test_hdr.avs

Stream #0:0: Video: rawvideo, 1 reference frame (Y3[11][10] / 0xA0B3359), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn

>ffmpeg -hide_banner -loglevel 40 -avisynth_flags none+matrix -i test_hdr.avs

Stream #0:0: Video: rawvideo, 1 reference frame (Y3[11][10] / 0xA0B3359), yuv420p10le(bt2020nc/unknown/unknown), 3840x2160, 25 fps, 25 tbr, 25 tbn

>ffmpeg -hide_banner -loglevel 40 -avisynth_flags all -i test_hdr.avs

Stream #0:0: Video: rawvideo, 1 reference frame (Y3[11][10] / 0xA0B3359), yuv420p10le(tv, bt2020nc/bt2020/smpte2084, progressive, left), 3840x2160, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn