AviSynth and mp4

[chris319]

I'm lost.

I'm trying to use avisynth to modify an mp4 file.

I've downloaded FFmpegSource. I have unzipped the downloaded file into its own folder. Now what?

The documentation isn't doing it for me.

How do I operate on an .mp4 file using avisynth and FFmpegSource?

Thank you.

[poisondeathray]

I'm lost.

I'm trying to use avisynth to modify an mp4 file.

I've downloaded FFmpegSource. I have unzipped the downloaded file into its own folder. Now what?

The documentation isn't doing it for me.

How do I operate on an .mp4 file using avisynth and FFmpegSource?

Thank you.






1) install avisynth

2) most plugins such as .dll's will autoload if you place them in the plugins folder. In your case it's ffms2.dll so place that into the avisynth/plugins folder

or you can explicitly load plugins in the script with LoadPlugin("PATH\whatever.dll")

3) create a script. You can use notepad or a script editor like avspmod . You can preview with avspmod or vdub. To do it in notepad you would open a text file, type the following (change paths and names to match the file), save it, rename the extension from .txt to .avs
e.g
myscript.avs


FFVideoSource("PATH\video.mp4")

[chris319]

Doesn't work.

Here is the line of code I'm using:

FFVideoSource("D:\Videos\2017-09-23 11-44-44.mp4")

ffms2.dll is in the plugins folder. Windows media player opens and gives an error message -- can't find FFVideoSource.

This doesn't work either:
LoadPlugin("D:\Programs\AviSynth\plugins\ffms2.dll")

WMP can't load the dll.

This code does work:

Version()

[poisondeathray]

Do you have the correct matching version(s) ?

ie. if you have avisynth 32bit, you would use the 32bit version. If 64bit, use the 64bit version

Which ffms2 version are you using ? Some versions are "C" plugins and they need to be loaded differently with LoadCPlugin

[hello_hello]

Avisynth is obviously working but are you using 32 bit Avisynth & 32 bit ffms2 or 64bit and 64bit? (edit: whoops, poisondeathray beat me to it)

"Can't find FFVideoSource" sounds like an Avisynth complaint and WMP is just displaying it. Is the error message actually "no function named FFVideoSource"?

I'd try AvsPmod (https://www.videohelp.com/software/AvsP) rather than WMP. Not that it'll magically fix the problem, but it'll make creating/previewing scripts a bit easier.

[chris319]

I had to change to the 32-bit dll.

This did not work:

#LoadPlugin("D:\Programs\AviSynth\plugins\ffms2.dll")

Next is to try to get avisynth to actually do something.

[chris319]

OK, this code works, playing back on WMP.

DirectShowSource("D:\Videos\2017-09-23 11-44-44.mp4")
FlipVertical


This code doesn't:

ColorYUV(off_y = 216)


This returns "Windows Media Player cannot play the file":

FFVideoSource("D:\Videos\2017-09-23 11-44-44.mp4")

This code complains that it's not a YUV file:

ColorYUV(gamma_y=128, off_y=-16, off_u=5)

MediaInfo reports:

Color space : YUV

Still lost. The docs are not helping.

[poisondeathray]

What is the FlipVertical for ?

If you had to "flip" it, that suggests your directshow filter chain is probably serving RGB (it's being inadvertently converted somewhere). That probably also explains why coloryuv doesn't work for you. You would have to "fix" your directshow system . Or better get some other more reliable source filter working like ffms2 or lsmash. Directshow is prone to many problems like the one you're having now, also other issues like frame accuracy

You can check with info() to see where you are at, after each stage


DirectShowSource("D:\Videos\2017-09-23 11-44-44.mp4")
info()

[poisondeathray]

If wmp can't play the avs directly, likely you need to match x86 vs. x64 (if wmp version is x64, it can't "see" an x86 script)

I would try avspmod (again use x86 or x64 depending on which version you're currently using) , because you're going to want to use the preview and edit and probably look at some monitoring functions like waveform (histogram) . It also has built in color picker that can read off RGB and YUV values

I have both x64 and x86 versions concurrently installed on one of my machines, but the x64 version is avisynth+ . But the point is x86 programs can only "see" x86 scripts and same with x64 programs and scripts

[chris319]

Here is all I'm trying to do:

1. Raise the luminance levels by 16

2. Hard clip luminance levels at 235. Yes, hard clip. I don't want to adjust the luminance here by lowering the gain; just a hard clip. I don't want to reduce 255 to 235. Just clip off the video, viz.:

if Y > 235; Y = 235

Nothing fancy.

The .mp4 file was created by OBS.

[poisondeathray]

Here is all I'm trying to do:

1. Raise the luminance levels by 16


More than one way to do this, but easy way is
coloryuv(off_y=16)



2. Hard clip luminance levels at 235. Yes, hard clip. I don't want to adjust the luminance here by lowering the gain; just a hard clip. I don't want to reduce 255 to 235. Just clip off the video, viz.:

if Y > 235; Y = 235

Nothing fancy.

The .mp4 file was created by OBS.

Again, more than one way , but easy way is limiter to set min/max luma . I left min at zero, but you can change to whatever you want clipped. You can visualize the pixels getting zapped by using the show option
http://avisynth.nl/index.php/Limiter


limiter(min_luma=0, max_luma=235)




Are you sure you want to do this? What is the reason?
A YUV MP4 file created by OBS would likely already have normal range

[chris319]

easy way is
coloryuv(off_y=16)


Not so easy. MediaInfo reports YUV. avisynth/DirectShowSource say it is not, no can do.

Are you sure you want to do this? What is the reason?

Yes. I want the luminance to be 235 or less without changing the middle tones as would happen if I applied a gain factor.

The .mp4 file I'm using for testing came from OBS. Other files will likely come from a camcorder with no OBS involved. Here is what I use to check camcorder video levels, along with Elgato Cam Link and a wavform monitor program I have written.

https://www.bhphotovideo.com/bnh/controller/home?A=details&O=&Q=&ap=y&c3api=1876%2C%7Bcreative%7D%2C%7Bkeyword%7D&gclid=Cj0KCQjwr53OBRCDARIsAL0vKrNyrJFVkiOxMaZpC9sXBH19eDPaGSVzus4uwafQcnWglgFbEnmgsJ0aAkdQEALw_wcB&is=REG&m=Y&sku=813250

[poisondeathray]

[QUOTE=chris319;1819525]Not so easy. MediaInfo reports YUV. avisynth/DirectShowSource say it is not, no can do.

[quote]

I already answered this one. Very likely it is YUV, but your DirectShow subsystem is screwed up. You can verify with info() to see what the source filter output colorspace and a bunch of other information

I asked which version of ffms2, but you never replied. Exactly where did you download it from ? I'm not asking for fun or to strike up a conversation; there are certain versions that are better than others. Also, there are "C" versions that require LoadCPlugin (not LoadPlugin) . l-smash is actually preferred for most types of MP4 (at least for me), because it doesn't require an index

DirectShowSource() relies on your system installed and configured directshow filters. So it's very unreliable and therefore questionable accuracy. The decoder configuration akes a difference. If decoder "A" outputs full range, but decoder "B" ouputs standard range, you're in trouble. You might get something on one computer but completely different on another. Your flipvertical strongly suggests you're actually serving RGB - you can check with info() . To configure your directshow system , You can use preferred filter tweaker, along with graphstudio. Likely you have the microsoft decoder with the highest merit. You want something else like LAV or FFDShow . But directshow is a "last resort" type source filter. Prone to many problems. For the most part ffms2 and lsmash are much more consistent in those regards

If you have 32bit avisynth, 32bit ffms2, open it up with 32bit vdub or mpchc or avspmod for now. Ditch WMP for now because there are too many other variables (such as directshow configuration) when you are starting out

You don't *have* to use avisynth. You can do what you want with other programs. But avspmod is nice because you can preview, assess and visualize with various scopes, RGB/YUV pickers, and edit scripts at the same time - and it's free

>90% of consumer camcorders record in the 16-255 range. The majority of those actually have usable data in the 235-255 range.

[poisondeathray]

If you want to avoid avisynth (or maybe revisit it later when you have more time) and do the equivalent in ffmpeg-ese , you can do it with lutyuv

This lifts Y +16, then hard clips 16-235.


-vf lutyuv=y='clip(val+16,16,235)'

[chris319]

Flipping the video is not mandatory. It was done to demonstrate that avisynth is working. That line has since been commented out.

ffms2 came from here: https://github.com/FFMS/ffms2

My camcorder is in the 10% that puts out blacks at 0.

[poisondeathray]

Flipping the video is not mandatory. It was done to demonstrate that avisynth is working. That line has since been commented out.

ffms2 came from here: https://github.com/FFMS/ffms2

My camcorder is in the 10% that puts out blacks at 0.

Is it ffms2-2.23.1-msvc.7z ? Did you unzip the 32bit version?

If coloryuv line complains that video is not YUV, then the video being served is not YUV . Still the most likely explanation is DirectShow has converted it to RGB behind your back (reason #27 not to use DirectShow)

Or, maybe the video isn't YUV ? IIRC You can setup OBS to record RGB

What model is your camcorder ? Some record full range (with full range flag) . So it depends on how you configure the decoder and what software you are using, and if you obey the flag or not. For example, many canon DSLR's do this , but they are supposed to be clamped (not clipped) .

[chris319]

The camcorder is a Canon Vixia HF R800. No tweak points that I can find for video levels, aside from some consumer-y gimmicks for shooting scenes at the beach, etc.

I seem to be making some progress with ffmpeg.

[poisondeathray]

The camcorder is a Canon Vixia HF R800. No tweak points that I can find for video levels, aside from some consumer-y gimmicks for shooting scenes at the beach, etc.

I seem to be making some progress with ffmpeg.



Yes - many consumer canon models have limited manual controls . But if it's like 99% of other canon consumer camcorders, you probably want to adjust the clip to Y~16-235, CbCr~16-240 ; not clip it. You probably want to "clamp" it, then make some other minor adjustments

eg . If you had some bright highlights , they are going to be lost if you shift Y+16 "blindly" and then clip it. Instead you should "recover" the highlights because 90% have usuable information in the 236-255 range, and 100% will have usable information in the 219-235 range - which you just clipped

I would recommend checking first to see how much is "usuable data" in the various ranges.

Instead of linear adjustments, you could also do other manipulations, non linear adjustments. e.g. preferentially bring up shadows, but not clip the highlights.

As much as avisynth is "clunky" to use for this (compared to real grading software), ffmpeg is even "clunkier". When you use avspmod, you can see and check the waveform / various scopes at the same time to fine tune your adjustments

[chris319]

I will check out avspmod. I do need to see this on a scope to make sure I'm actually getting what I want.

There are two ways to handle this with ffmpeg:

-vf lutyuv=y='clip(val+16,1,235)'

-vf lutyuv=y='clip(val*0.8588235294117647+16,1,254)'

Where the gain factor = (235 - 16) / 255. The latter is probably preferable. The clipping at 1 and 254 keeps video data out of the sync.

Will avspmod have an easier time dealing with .mp4 files? I spent many hours yesterday trying to get avisynth to work with .mp4 and got nowhere with it.

[poisondeathray]

avspmod is like a GUI of sorts for avisynth. So if you can't get avisynth working , forget about avspmod. If you can't get ffms2 working you can try l-smash. You're going through what is affectionately known as "dll hell" :) . Everyone goes through it at some point with avisynth



-vf lutyuv=y='clip(val*0.8588235294117647+16,1,254)'

Where the gain factor = (235 - 16) / 255.



I think something is off with that ffmpeg multiply factor; it's clamping too much compared to a levels(0,1,255,16,235,false) , so levels are washed out , low contrast . I'm getting about 30-218 on a full range clip

EDIT: I think I might know why - my test clip is flagged full range, so I think ffmpeg might be double clamping... I'll run some more tests. Or I guess it depends if your camera flags it or not. This canon does

[lansing]

Ok here's a installation guide start from scratch:

- get the AVS+ installer here (https://www.dropbox.com/sh/oxx5cm9hkbpj5oz/AAD0QBnTlczv7xW3jEdSjenHa?dl=0), r2294, and install it
- get the latest version of avs+ here (https://github.com/pinterf/AviSynthPlus/releases) and replace the corresponding files
- get the latest version of ffms2 here (https://forum.doom9.org/showthread.php?t=174469) and put the dll into your 32bit/64bit plugin folders
- get avspmod for script preview

Call the function like this
ffms2("test.mp4")

No more calls from ffvideosource.

For the blown highlight recovery, I guess you're probably looking for highlightlimiter (https://forum.doom9.org/showpost.php?p=1574765&postcount=60).

[poisondeathray]

Yes, ffmpeg automatically "clamps" certain output formats when it sees the full range flag unless you override it with -vf scale with input/output levels. It doesn't clamp -c:v libx264 , so levels are correct with a full range YUV input clip with that lutyuv, but it passes the full range flag - which isn't good . Because this is now a normal range clip, flagged full range

Did you want to upload a small sample clip? this might be completely irrelevant for you as I'm testing from a different camera model

[chris319]

if you can't get avisynth working , forget about avspmod.

You just saved me a couple of days of frustration.

My needs are so modest, I think ffmpeg will suffice -- and it works.

Check my math. Why wouldn't that work?

[poisondeathray]

Check my math. Why wouldn't that work?

Math works and it makes sense if it's a full range clip

But it's a ffmpeg issue with flags. It might or might not be applicable for you, but there can be some other issues stemming from that

[chris319]

Here is what I get from MediaInfo:

General
Complete name : D:\Archive\ffmpeg\zeranoe\CurrentVersion\outfile.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 262 KiB
Duration : 10 s 444 ms
Overall bit rate : 206 kb/s
Writing application : Lavf57.76.100

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 4:4:4 Predictive@L3.1
Format settings, CABAC : Yes
Format settings, RefFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 10 s 444 ms
Bit rate : 125 kb/s
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.005
Stream size : 160 KiB (61%)
Writing library : x264 core 152 r2851 ba24899
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=4 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range : Full

Why would it be a bad thing if the color range were set to full?

[poisondeathray]

Why would it be a bad thing if the color range were set to full?

Because it's wrong . You just scaled the clip to normal range so there is mismatch. The problem is some software/hardware won't display it properly . Ideally , you would want standard range, properly flagged. Or even no flags are better than wrong flags

Also that output was 4:4:4, so I'm guessing some other issues, unless that was desired. Because a source camera clip from consumer canon model won't shoot 4:4:4

I'm guessing your canon camera had similar characteristics as my test clip. But if you upload a sample I can double check

[chris319]

The media info data is for the output of ffmpeg, not the raw file from the camera.

Let's move this discussion.

https://forum.videohelp.com/threads/385078-Adjusting-H-264-Levels-from-mp4#post2496991

[poisondeathray]

The media info data is for the output of ffmpeg, not the raw file from the camera.

Yes, that's what I mean. The actual levels should be fixed, but it's flagged full range , which can cause problems. Ideally it should say "limited" or nothing at all. For example if you uploaded that to youtube, it would clamp it again because of the flag (double clamp)

Also, presumably you'd want to keep the same as input 4:2:0, not chroma upsample to 4:4:4

There should be a simple workaround in ffmpeg, just specifying the input and output levels


How do I upload an .mp4 file here?
You can upload to a free hosting site such as zippyshare.com, mediafire.com, even google drive