Video player Shaders guide (MPC-HC, mpc-BE, mpv)

[butterw2]

Video player Shaders (unofficial guide)

Video Adjustment shaders Pack (dx9, dx11 hlsl and mpv glsl.hook): [A-Pack] v1.50 Released (2023-11)
Contrast adaptive Sharpening (AMD CAS) shader (dx9/dx11).
Filmgrain Shaders grain.hlsl (dx9/dx11), mpv


In addition to video player shaders, this topic covers configuration, user interface and performance of open-source video players available on Windows.
MPC-HC MPC-BE, mpv, mpv.net are feature packed video players that offer great features and performance, together with extensive configuration options.

Post index:
# Comparison of shader features in video players
# Shaders Folder, Paths & extensions
# New Shader features in Mpc-hc (v1.9.8.65)
# Custom hotkeys, Seeking, Chapters, Playlist
# Screenshots
# Comparison of video player features (Win10)
# Mpc Command Line Switches
# Video Player installation https://forum.doom9.org/showthread.p...73#post1934473
# Color mapping with Shaders https://forum.doom9.org/showthread.p...82#post1935482
# (custom) windowed zoom: 150%
# Time display formats, ex: 01:25 / 53:15

MPC video-renderer, Mpc-vr
# Shader support in mpc video renderer (mpc-vr)
# Surface/texture format

MPV
# mpv (windows): my tips New
# Mpv shader basics
# mpv hotkeys
# mpv built-in scalers (2023-10) new Defaults !
# mpv Auto-Profiles
# How to change settings/properties at runtime with mpv
# default input.conf. --no-input-default-bindings
# custom OSD in mpv
# mpv script: avg-bitrate.js
# more mpv custom OSD scripts
# mpv-android
# restart-mpv
# custom context menu
# custom overlay: modern progress bar
# mpv scripts: lua or js ?
# mpv script: short time format OSD, btime.js

# Cartoon Shaders https://forum.doom9.org/showthread.p...55#post1928755

Videohelp:
# What shaders should I use in potplayer?
# Pixel-shaders for video playback (.hlsl, .glsl)
---

Fix Mpc-hc issues with evr-cp (dx9) on intel Xe driver (used by 11th gen and later intel igpu, ex: intel uhd730)
Issue: Broken seeking after reaching end of file, repeat doesn't work. Affects all versions of mpc-hc.
Fix: rename mpc-hc64.exe to mpc.exe.

Mpc-hc + Shaders install (with default dx9 EVR-CP renderer) unofficial guide
32 vs 64 bit program version doesn't matter for shaders.

Mpc-hc installs the default shaders in the player directory Shaders subfolder. They are not required to run mpc-hc (they can be modified/deleted as required). It is also possible to download shader files available on the internet or write your own. Files with .hlsl extension in Shaders subfolder are listed in `Option->Shaders` submenu.

Install directory does matter.
To be able to modify shader parameters you need to have write access to the directory where the shader.hlsl file is located (by default the Shaders subfolder).
Also to have the option to use mpc-hc.ini, mpc-hc must have write access to the Install directory (if not the option is greyed out). This will not be the case for a standard user on Win10 by default with a `c:\Program Files\MPC-HC` install. To get around this, you need to either re-install to a user directory or change folder permissions. For testing, the simplest solution is to copy a portable version + mpc.ini in its own directory (no-install required).


How to get corresponding Shaders Menu when you install Shaders files ?
To be able to apply Effects quickly, they need to be saved as Shader Presets. You can then change Effects through the Shaders right-click Menu
Shader presets can be created/saved by using the `Select Shaders...` menu option, but by editing mpc-hc.ini we can get a ready to use Shaders Menu directly.



Mpc-hc Shader Menu mod guide:
Mpc-hc settings are stored either in the registry or in an .ini file, based on setting `Mpc-hc->Options->Player->Store settings in .ini file.`. This option will be greyed out, if mpc-hc doesn't have write access to the install folder.

Editing mpc-hc.ini (Close Mpc-hc before you start. It is advisable to make a backup copy before making any changes)

With a text editor, search for the Shaders\Presets section. Here is an example of what you might have:
[Shaders\Presets]
0= Disable
1=Sepia
2=Luma+Grain
3=Pixelate
PostResize0=
PostResize1=
PostResize2=.\LumaSharpen.hlsl;.\PS_Noise.hlsl
PostResize3=
PreResize0=
PreResize1=.\Sepia.hlsl
PreResize2=
PreResize3=.\bPixelate.hlsl

You can directly edit preset names, Pre and PostResize shader chain filepaths. Be careful that all numbers must remain available: deleting 1=Sepia would cause issue here !
Shaders Menu display order: Preset names are sorted alphabetically. To control the display order, preset names could be numbered (00, 01, etc.).
For more advanced users, Shader Presets can be used to setup Multi-Pass shaders.


Modifying shader files
.hlsl files contain the Shader function and parameter values. In most cases, the effect has parameter values that can be modified. This can be done with any text editor (I use Notepad++, a multitabbed editor which offers c-syntaxic coloring). After saving the changes, they are applied instantly to the video playing.
In the case of LumaSharpen.hlsl which was developed for SweetFX game shaders pack, the default parameter values are not those recommended in this forum for video content playback.
If you want to switch between different parameter values for the same shader without editing, you can create a new shader with the desired parameter values, ex: LumaSharpen_3-1.20-0.015.hlsl. With this approach the Shaders subfolder (and the shader list) can quickly become overcrowded, so it is important to know which shaders you want to use, together with the associated parameter values.

[v0lt]

I think there isn’t enough shader compilation checking.

For DirectX 9

fxc.exe /T ps_3_0 /Fc "Shader9.txt" "Shader9.hlsl"

For DirectX 11

fxc.exe /T ps_4_0 /Fc "Shader11.txt" "Shader11.hlsl"

[butterw2]

I haven't tried the external dx11 renderer so far, but I understand it requires dx11 shaders, which have a slightly different syntax than dx9 shaders.

Doesn't the compiler have a backwards compatibility mode (fxc /Gec ?) to enable use of dx9 shader files in dx11 ?

All the currently available third party .hlsl shaders available online were written for dx9.

[butterw2]

Optimized 2-pass Gaussian Blur (9-tap) shader:
bShaders\blurGauss.hlsl >> blurGauss_Y.hlsl
the kernel is hardcoded so you can't adjust parameters. Run it multiple times to achieve a stronger effect (ex: 5x).

below are Blur Gaussian and Blur Gaussian (5x) shader presets for mpc-hc.

mpc-hc.ini:
[Shaders\Presets]
0= Disable
1=Blur Gaussian
2=Blur Gaussian (5x)
PostResize0=
PostResize1=
PostResize2=
PreResize0=
PreResize1=.\blurGauss.hlsl;.\blurGauss_Y.hlsl
PreResize2=.\blurGauss.hlsl;.\blurGauss_Y.hlsl;.\blurGauss.hlsl;.\blurGauss_Y.hlsl;.\blurGauss.hlsl;.\blurGauss_Y.hlsl;.\blurGauss.hlsl;.\blurGauss_Y.hlsl;.\blurGauss.hlsl;.\blurGauss_Y.hlsl

For a near gaussian blur with an adjustable parameter, use boxBlur (3x) based on 2pass blurMean.
A multipass Kawase blur, also can approximate a gaussian depending on parameters.
For best performance with Heavy bluring, Dual-Kawase downsizes the video, blurs, and resizes-up.

[butterw2]

Mpc-hc and mpc-be have a "View>Pan&Scan" zoom-crop built-in feature. This transformation happens before user shaders.

Editing PnS Presets: specify center (x,y) and zoom (x, y).
- center (0.5, 0.5): middle of the screen
- zoomX and zoomY should be equal to keep the source Aspect Ratio. 1: no magnification.

The most useful preset is probably "Zoom to widescreen" which will crop horizontal and vertical borders on the corresponding 4/3 source material when displayed on a 16/9 screen.

- `Numpad 5` hotkey: resets PnS zoom (also displays source AR). PnS zoom is also reset when you close the player.
- You can manually adjust zoom using Numpad 9(+) and 1(-) hotkeys.


Observations:
- Center of Screen PnS fixed zoom works well, whatever the source Aspect Ratio.
- I initially tried to display only part of the source image with Pan&Scan (ex: fill the screen with first quarter of frame), but found pre-resize shaders more capable for this use case.
The issue with PnS (beside the confusing x, y values) is black bars on resize, when the source doesn't have screen 16/9 AR (ex: 21/9 film content, 9/16 vertical video).

Quarter zoom PnS presets: x, y values (zoom: 2, 2), works well for 16/9 input.
- top left Quarter: 1, 1, - top right: 0, 1.
- bottom left Quarter: 1, 0 - bottom right: 0, 0.

Fs2xRightSide PnS preset could be used for right side offset 2x zoom for non 16/9 landscape.

in mpc-hc.ini: [Settings\PnSPresets]

Preset0=Widescreen zoom,0.500,0.500,1.333,1.333
Preset1=Quarter Top-L,1,1,2,2
Preset2=Fs2xRightSide-L,0,0.5,2,2

You can generally combine user pixel shaders with any mpc-hc zoom feature (including Right-Click: Video Frame > xxx and Zoom > xxx).

"Video Frame > Touch window from outside" is a centered fullscreen crop mode based on the input video Aspect Ratio. Fills the window/screen by zooming and cropping the video.

[butterw2]

Linear sampling is a performance optimization technique for gpu pixel shader. The idea is to reduce the number of pixel (texture) fetches required by an algorithm by sampling between pixels and exploiting hardware linear interpolation.

a 2-pass pre-resize shader test was performed:
https://github.com/butterw/bShaders/...linearSampling
- pass-1 generate a black and white stripe test motif,
- pass-2 use linear sampling

Expected: interpolation, colored Red
Result: no interpolation (Nearest Neighbor). If the offset is set to 0.5 pixel: produces some rounding artefacts !



Software and hardware setup
- mpc-hc v1.9.6 + EVR-CP. Also tested in mpc-be
- old intel igpu (hd4400), not the latest Win10 + drivers


## Linear Sampling (in hlsl): tex2D(s0, tex + k*p1) where k is non integer
the effect of Nearest Neighbor sampling is to round k.x, k.y to the nearest integer. If k.x or k.y is a half, you then get some rounding artefacts.
While it certainly is possible to get rid of the rounding artefacts, they are a symptom of the larger problem: Linear Sampling not being supported, which has major implications for the math analysis.

Linear sampling is exploited by perf optimized convolution kernel shaders (such as: Luma Sharpen, gaussian blur, etc.). If it isn't supported, the shaders still produce an output, but a suboptimal one. The provided test allows for the user to test whether linear sampling is actually supported.

Edit (linear sampling works in mpv): the same test using rgb32 png input was performed on the same configuration with a glsl shader (mpv video player, vo=gpu), and I'm getting the expected result there !
https://github.com/butterw/bShaders/...linearSampling

Edit2 (hw linear sampling isn't supported in mpc-hc/be): hw linear sampling also not working as expected in mpc-hc/be with mpc-vr as video renderer in dx9 and dx11 modes.
https://github.com/clsid2/mpc-hc/issues/748
This impacts shaders such as LumaSharpen.hlsl

[VictorLS]

butterw2
Can you help us (SAT amateurs) in writing HLGtoSDR(BT.709) hlsl shader https://forum.doom9.org/showthread.php?t=176909 i.e. adopt libplacebo to MPC-HC's shader we can use in SmartDVB - quality of present shaders is bad - but VLC and MPV players give good enough quality while playing HLG files on SDR monitor?

[butterw2]

If glsl fragment shader code exists for the desired color transformation that would mostly autotranslate to hlsl usable in mpc-hc/be.

[VictorLS]

I know but result of that translation is bad enough or some 'know how' must be known to get it working well.
Besides - I don't really sure libplacebo (only?) do desirable color transformation (HLGtoSDR) or VLC uses something else - I'm not programmer at all and moreover I'm not shader programmer (
Here's somehow working HLGtoSDR shaders https://transfiles.ru/svler - Convert HDR to SDR.hlsl in HDR folder work well enough but for ordinary HDR files like 23_Samsung_HDR_Wonderland.mkv (97 MB) https://yadi.sk/i/ZgkOWjcEwWQNkQ - not for HLG files like 20190831-160457_RTL UHD.mkv (135 MB) https://yadi.sk/i/SRloXWrOP-L-xA
I even don't know that (not in HDR folder) shaders was taken from libplacebo or somewhere else
PS. Here's long time (more than 4 minutes) test files https://forum.doom9.org/showthread.p...34#post1919134

[butterw2]

I'm afraid, my experience on the subject of HDR to SDR tonemapping is rather limited.
mpc-be does some tonemapping by default (using hlsl shaders I believe). If I understand you correctly, you claim that the method used by vlc and mpv gives more accurate results ?

If there was a mpv glsl .hook file, that might be a better starting point for a port than libplacebo (c file with inlined GLSL).
I've tried the provided "Convert HDR to SDR.hlsl" shader in mpc-hc. The colors seem oversaturated for my taste (while the colors are washed out without it). The processing is not that complex (~100 lines of actual code), so it should be achievable to get the shader part working if a reference glsl implementation exists.

Regarding the provided sample files:
- 20190831-160457_RTL UHD.mkv is UHD-50 hevc 10bit HDR 41Mbps
- 23_Samsung_HDR_Wonderland.mkv UHD hevc 10bit HDR 22.5Mbps
Without considering the issue of tonemapping, these files do not play smoothly on an old pc/integrated graphics + 1080p SDR screen like mine.

For testing purposes, short samples are probably fine, I would suggest downconverting the most relevant ones to 1080p, reducing the bitrate 10/20x, maybe remove the audio, provide screenshots (with frame timestamp/frame number) of what you regard as desirable (from mpv, avisynth) vs what was achieved and host this with the code on a server that will not delete your files.

[VictorLS]

Thanks for reply.

I'm afraid, my experience on the subject of HDR to SDR tonemapping is rather limited

It's a pity but I believe in you
Btw I have SDR (BT.709) monitor too - that's why I want to find good HLGtoSDR shader but couldn't yet (

20190831-160457_RTL UHD.mkv is UHD-50 hevc 10bit HDR 41Mbps

Not only ordinary HDR, but HLG same time - that's why working with ordinary HDR video like

23_Samsung_HDR_Wonderland.mkv UHD hevc 10bit HDR 22.5Mbps

Convert HDR to SDR.hlsl shader gives good result but with HLG 20190831-160457_RTL UHD.mkv oversaturated.

mpc-be does some tonemapping by default (using hlsl shaders I believe)

Yes, you're quiet right (btw MPC-HC must output to madVR or use Convert HDR to SDR.hlsl shader with any other renderer to achieve such results - am I right?)

If there was a mpv glsl .hook file, that might be a better starting point for a port than libplacebo (c file with inlined GLSL)

From MPC-BE developers I've heard mpv does it on a fly so they are couldn't extract using by mpv shader while playing HLG file (btw they are looking at my researching for HLGtoSDR shader and ready to implement it in MPC-BE like now HDR if result will be positive). Besides quality of tonemapping depends of version of mpv - older than August 2019 are better (newer gives some rose instead of red) on my own so I'd prefer shader extracted from VLC

For testing purposes, short samples are probably fine

I agree in your case - you can find them in the bottom of first post https://forum.doom9.org/showthread.php?t=176909

I would suggest downconverting the most relevant ones to 1080p

I guess it's wrong way because HLG metadata can be lost during any conversion so see first post for your other wishes

[butterw2]

Tone mapping is required to correctly display any HDR encoding (typ 1080p Hevc 10bit HDR encode from UHD bluray) displayed on a Standard Dynamic Range computer screen (almost all current monitors are non-HDR).

mpc-hc doesn't do any tone mapping by default: so colors look washed out. If you enable the provided Convert HDR to SDR.hlsl shader, the results seem quite watchable for HDR content. It's also a reasonably lightweight shader (1 texture, 44 arithmetic).
Edit: the code likely comes from mpc-be and has 2 tuning parameters (LuminancePeak_Gain, Gamma).
Such a shader can improve the viewing of a washed-out color video (ex: encode of an HDR source to SDR with no tonemapping applied, or undersatured SDR video). It shouldn't be used otherwise.

mpc-be and vlc come with built-in SDR Tone mapping, which is applied automatically (there doesn't seem to be a setting to deactivate it !).

The samples (RTL title screen) in your first post here https://forum.doom9.org/showthread.php?t=176909
seem like a good starting point regarding the specific case of Hybrid Log-Gamma (HLG) HDR used in UHD satellite broadcasts .

- 23_Samsung_HDR_Wonderland.mkv (UHD hevc 10bit HDR 22.5Mbps)
-20190831-160457_RTL UHD.mkv (UHD-50 hevc 10bit HLG HDR 41Mbps): quite specific sample because it contains a lot of white smoke. I actually prefer no tone mapping on this sample vs oversaturated result.

Also, on my PC performance is awful with VLC with these 2 samples.

[VictorLS]

Tone mapping is required to correctly display any HDR encoding (typ 1080p Hevc 10bit HDR encode from UHD bluray) displayed on a Standard Dynamic Range computer screen (almost all current monitors are non-HDR)
mpc-hc doesn't do any tone mapping by default: so colors look washed out. If you enable the provided Convert HDR to SDR.hlsl shader, the results seem quite watchable for HDR content. It's also a reasonably lightweight shader (1 texture, 44 arithmetic).

So, I was right when wrote "MPC-HC must output to madVR or use Convert HDR to SDR.hlsl shader with any other renderer to achieve such results"

mpc-be and vlc come with built-in SDR Tone mapping, which is applied automatically (there doesn't seem to be a setting to deactivate it).

You're right.

The samples in your first post here https://forum.doom9.org/showthread.php?t=176909
seem like a good starting point regarding the specific case of Hybrid Log-Gamma (HLG) HDR used in some satellite broadcasts(RTL title screen).

Some people said this is bad point so I hard work to write red-flagged F1 to someone do right shader with colorimeter
SDR FullHD H.264 20190511-130113_ARENA HD_flag.ts (38 MB) https://yadi.sk/i/HiuZ5Rb2RKkc0g
HLG 4K H.265 20190511-152944_RTL UHD_flag.ts (67 MB) https://yadi.sk/i/985DVw34OHQrrA

-20190831-160457_RTL UHD.mkv (UHD-50 hevc 10bit HLG HDR 41Mbps): quite specific sample because it contains a lot of white smoke.

I've uploaded that to see how new HLGtoSDR shader shows grass - it's looking ugly now with or without tonemapping in ordinary DirectShow players.

I actually prefer no tone mapping on this sample vs oversaturated result.

You're right but I'd want get VLC (OpengGL output in Win7 or D3D11 output in Win8-10) colors with HLGtoSDR shader

Also, on my PC performance is awful with VLC with these 2 samples.

and weak PC can't stop experiments with colors because of watching mostly in pause mode

[butterw2]

# Comparison of user shader features in the main video players for Windows

To Everyone: Please point out any mistakes/changes, so I can update this guide as needed. The reason for this thread is that the available documentation is sparse and mostly buried in very old/long threads.
You don't need to be a C++ dev to customize or write pixel shaders.

## directX hlsl shaders:
VLC v3.10 in dx9 video output mode
- only one user shader (post resize) !
built-in effects/filters only work in dx11 mode !
shader is set in config options > advanced, but can also be set via command-line
! code: the output alpha channel must be != zero, otherwise black screen output.
shader modification: stop/start video.

MPC-HC v1.9.6 (EVR-CP) -- Recommended for shader dev, New features added in dev build --
- Configurable Shader Presets: a named/saved user defined shader chain.
- in menu Select shaders (uses Options shortcut "O")
- switch shader presets via contextual Menu in fullscreen mode
- pre and post resize pixel shader chains
- multipass shaders possible
- Shader modification: new parameters are applied immediatedly (! or next frame in the case where you empty the shader chain).
- Compiler output with Debug Shaders (+ shortcut)
- time in seconds since application start available and updated every frame.

- KMPlayer 4.0.7.1 can fully operate with shaders when paused


MPC-BE v1.5.4 (EVR-CP)
- ! No Shader Presets: single pre and post resize shader chains
- Shortcuts to enable/disable pre and post resize shader chains (CTRL-P, CTRL-ALT-P)
- "Save output image" Screenshot includes shader output (Shift+F5)
- ! Shader modification: close/restart player.
- ! unlike mpc-hc, Alpha Channel is initially set to 1 (because of this, some shaders that assume initial alpha is 0 will not work properly without correction).


---
## glsl fragment shaders intro here

Mpv .hook (vo=gpu, gpu-api=d3d11)
set via glsl-shader in config file mpv.conf.
(Shift-I, Shift-2) to display the processing chain. Hotkeys can be added in input.conf to switch individual shaders, disable/enable shaders or list active shaders.
- Output is a blue-screen if a shader has a syntax error. You need to restart player to modify shaders.
- user shaders with custom hook points: source shaders (Luma, Chroma), chroma upsampling, custom resizers, pre and post resize RGB.
- multipass shaders in a single file, can downsample textures, can re-use the output of previous passes.
- complex shader chain possible by saving textures.
- uniform variable "random" available (PRNG) and updated each frame
- compute shaders
! few .hook effect shaders currently available (vs hlsl, glsl), but easy enough to adapt .frag glsl to .hook

[butterw2]

# Video playback pipeline on Windows in Standard Dynamic Range (SDR, Video+Display)

Mpc-hc works in 8-bit by default.

0) Graphics Driver Settings (mostly disabled >> set to "Application Settings")
- features supported by gpu hardware.
- Output should be set to the same as your display (typ for PC: RGB full range)

1) Video File (container: .mp4, .mkv)
- Video (Codec: x264, x265)
- typ. YUV420 (chroma channels are subsampled), Gamma-Encoded 8/10bits,
- Resolution+Progressive+Framerate, ex: 720p30,
- color primaries and matrix: bt709 (HDTV)
- color range: Limited (Low: 16-235), (vs Full/PC: 0-255)
...streaming/bluray movies and TV shows are mastered in Limited range.
- Audio channels
- Subtitle channels
Mpc-hc/be Menu: File > Properties: MediaInfo.dll file information.

2) DirectShow player x86 or x64 (ex: mpc-hc, etc.)
- Features: window, play, seeking, configuration/Menu/Hotkeys, shaders, screenshots

Modern video players come with the decoders required to play any standard video without the need to install codecs/codec packs.
- DShow graph: Splitter > Decoder* > Renderer*
2a) LAV Filters - ffmpeg based DirectShow Splitter and Decoders (LAV Splitter >> LAV Video + LAV Audio)
MPC-HC/BE Menu: Play > Filters > Video Decoder
The decoder defines the color space at the renderer input (typ. same as source: YUV or RGB, 8, 10, 16bits)

2b) Renderer: DirectX 9 EVR-CP (Enhanced Video Renderer-Custom Presenter >Win8) + DXVA2 gpu hw Acceleration*
- default in MPC-HC

MPC-HC/BE Menu: View > Renderer Settings > Display Statistics (Ctrl+J) overlay
Input, ex: NV12, YV12
- Save Image... (Alt+I) input Screenshot
- YUV >> RGB Conversion
2c) RGB Processing Surface: int8, int10, float16
- Color Controls (Brightness, Contrast, Hue, Saturation)
- Zoom, Pan&Scan
- Rotation
- Pixel Shaders (gpu)
- RGBA float4(32, 32, 32, 32)
- Pre-resize user Pixel Shader chain
- Resizer (ex: bicubic A=-0.75, to 1080p)
- Post-resize user Pixel Shader chain (in Screen Space)
BackBuffer/Display ?

2d) Overlay (Alpha blended)
- Subtitles
- OSD, On Screen Display

MPC-BE (Shift+F5): Save Displayed Image (auto) - output Screenshot
Mixer Output ?

3) Output interface/cable: typ. hdmi
> RGB Monitor/HDTV (Resolution/Framerate, reported bit depth: 8 or 10bit), ex: 1920x1080, 60Hz, 8bit
- Settings on Monitor. (Brightness/Contrast/Gamma, RGB Range, etc.)
- SDR (sRGB), color: bt709.
- Gamma-decode

[v0lt]

Originally Posted by butterw2

Mixer Output ?

EVR Mixer usually requires a specific output format. See "DXVA Checker -> Processor device -> DXVA1/2 -> OutputFormat" for details. This is essential for AMD and Intel graphics cards to function properly. For Nvidia video cards, we may use some "unsupported" formats as output.

[butterw2]

# Playing a video file from .avs (Avisynth)
- Video players such as mpc-hc/be can play .avs files.
- Frameservers such as Avisynth can be used to process video (they are widely used pre-encoding).
So, can you process video files for realtime display using .avs and is there any benefit to do so ?

Avisynth processing is done primarily on the cpu vs pixel shaders, which use gpu processing.
- For many basic video related operations this is less efficient, even with multi-threading, but it also has fewer limitations.
- Avisynth scripts rely mainly on high quality existing internal/external functions/libraries (compiled .dll C++ plugins).


.avs Display Pipeline:
Video file (.mp4, ex: x264/x265 1080p, YUV420) >> .avs: source Filter(video_filename): Avisynth cpu processing: Output (ex: YV12 or RGB32)
>> Input of Video Player: Video Player processing (RGB), ex: gpu pixel shaders >> Monitor (RGB)

Performance:
- as expected, significant cpu overhead with an .avs script vs normal playing. Depending on the Avisynth filters used, realtime playback may however be achieveable.
- Avisynth filters can be used in complement to gpu pixel shaders.

Software configuration: Win10 x64
- DirectShow Video player: Mpc-be x64, EVR-CP renderer
- Avisynth+ v3.6.1 x64
- external Source plugin: LSMASHSource.dll x64, based on libavcodec, with gpu hw decoding support.
My tests were performed with Avisynth, but the more recent python script alternative Vapoursynth looks promising. It offers direct access to the python software stack (ex: numpy vector math, matplotlib plots, etc.).

## Limitations:
- LSMASHVideoSource / LSMASHAudioSource only support mp4/mov input ! Wider input file format support is available (ex: mkv) with LWLibavVideoSource, but input file must initially be indexed (ex: 10s) !
- video_filename is typically changed by editing the .avs script ! To avoid this, video_filename.avs can be coded so as to load video_filename.mp4 by default.
- On script syntax error, you only get "Cannot render the File" message in video player ! To get full error msg you need to use either AvsPmod, Virtualdub2 or AvsMeter.

Conclusion:
It is currently possible, but not very practical to play an avs script in a video player. This would probably only make sense if the same effect was not possible as a pixel shader and an equivalent player feature was not available.
Using a per-file script does mean file-specific processing settings are saved. Undesirable lossy re-encoding of a compressed source may be avoided.

Avisynth input can however be useful as a test tool for video:
- YUV-to-RGB colorspace conversion can be performed in Avisynth (with rec709 range expansion by default). In this case the YUV-to-RGB conversion at the video player input is bypassed.
- Resize frame
- Generate test-patterns (ex: ColorBarsHD).
- Go to a specific frame (vs Directshow video player: seeking is not guaranteed to be frame accurate !).
- Supports multiple video inputs. You could also use multiple clips from the same source (ex: could be used for a split-screen effect or to highlight differences)
- Load/process/save images (ex: lossless .png screenshots) with the same code.
- Output data files (ex: .csv)

[butterw2]

# Typical PC Display (Color Range) Pipeline :
Compressed video file (yuv420p8, Limited Range, Color range: Limited or untagged) >> mpc-hc Video player (Range expanded to Full Range at yuv-rgb32 conversion at renderer input)
>> graphics driver (rgb32 Output, Full Range) >> Hdmi >> Monitor (Full Range)

- Historically TV/HDTV have used limited range input vs PC monitor full RGB range.
- Most current displays should allow you to set the desired input range (if you change this value, you probably also need to adjust brightness, contrast, gamma for good results).
For accurate representation of grey levels and colors your monitor/HDTV and graphic card settings should match: Full-PC or Limited-Low.
Display range can be checked on screen with a pixel picker and a generated SMPTE ColorBarsHD test pattern input.

If the input video file is Full Range but lacks the correct video stream tag (Color range: Full) in the Mediainfo properties, mpc-hc/be will treat it as Limited range !
To avoid a clipped color levels output, you should correct the tagging, which can be done losslessly for x264/x265 using ffmpeg:

ffmpeg -i "input.mp4" -c:v copy -bsf:v h264_metadata=video_full_range_flag=1 "output_full.mp4"
(for x265): -bsf:v hevc_metadata=video_full_range_flag=1

More info about ffmpeg video bitstream filters here: https://ffmpeg.org/ffmpeg-bitstream-...4_005fmetadata
This can be used to crop a displayed video file without having to re-encode it for instance.

An out-of-range highlighting pixel shader such as https://github.com/butterw/bShaders/...ge/bHighL.hlsl can be used to highlight pixels that are out of the defined range.
Pixel shaders can of course be used to alter levels at playback, but this is not typically necessary.

[chros]

Originally Posted by butterw2

If the input video file is Full Range but lacks the correct video stream tag (Color range: Full) in the Mediainfo properties, mpc-hc/be will treat it as Limited range !
To avoid a clipped color levels output, you should correct the tagging, which can be done losslessly for x264/x265 using ffmpeg:
ffmpeg -i "input.mp4" -c:v copy -bsf:v h264_metadata=video_full_range_flag=1 "output_full.mp4"
(for x265): -bsf:v hevc_metadata=video_full_range_flag=1

I think this is excatly what I need for my "issue".
Do you know how to list all the video properties of a file with ffmpeg?
You can reply there as well, thanks!

[SirMaster]

Hi @butterw2

I see you have some experience in writing HLSL.

I am interested in writing some of my own shaders but I find myself pretty lost with the syntax.

I am a developer, so I have a lot of programming experience, but maybe you help me get off to a start.

I found this shader which works, but I would like to start by simplifying it.

https://pastebin.com/VMfh8tNL

This shader allows performing RGB color convergence adjustment (useful for 3 chip projectors).

However I notice that when I enable it, it seems to always be blending pixels which screws up 1:1 pixel mapping.

I think this is due to the fact that this shader allows for sorts of scaling and geometric adjustments of the color which I don't need at all.

I would like to modify the shader to only do simple whole pixel adjustments of the red and blue colors.

If you have time, do you think you can help me get started?