FFAStrans - FFMpeg Avisynth Transcoder

[FranceBB]

Hi,
I've been using Avisynth for years, but ever since 2016 we needed to automatize our workflows a lot as we had to encode too many files and we didn't have enough encoders, so manually encoding each and every raw file wasn't an option anymore.
Since then, I've started creating a few batch scripts and Avisynth scripts to automatize some common things based on the characteristics of every file, but then I came across a project that Steinar Apalnes was working on called FFAStrans and I switched to it.
Today I'm gonna talk about it and the power of open source, as I think that many broadcasters and production houses are paying a lot of money for softwares like Telestream Vantage, Harmonics ProCoder, Adobe Media Encoder, Selenio, AWS Transcoder and similar while Avisynth, ffmpeg and x264 can deliver better results and are also free.


User Interface:

https://i.imgur.com/2fSzFdQ.png


This is how the user interface of one of my main workflows looks like. It might seem a bit messy, but it's actually pretty clear.
Let's take a look at the Avisynth_in workflow which is meant to encode files in MPEG-2 XDCAM with the following specifications:

Video Codec: MPEG-2
Resolution: 1920x1080
Bitrate: 50 Mbit/s
Sampling: 4:2:2 yv16 planar
Framerate: 25i
Type: Interlaced TFF
Bit depth: 8bit

Audio Codec: PCM
Bit Depth: 24bit
Audio Channels: 8ch mono

The workflow starts with a watchfolder, then it starts indexing audio and video using FFMpegSource2, then it checks for the number of audio channels and basically always makes 8 channels (i.e if the original file has 1 ch, it duplicates it to all the 8 channels, if it has 2 audio channels, it duplicates left and right to the 8 channels and so on). Then, it applies loudness correction by making sure that it's 48'000Hz, 24bit and that the loudness is -24LUFS, then it checks for interlacing and it decides whether it has to deinterlace or not and eventually use bob-deinterlacing using yadif and so on. After that, it cheks the resulting framerate and it decides whether it has to apply a speed-up + pitch adjustment (23.976fps -> 25), or if it has to leave it as it is 'cause it's 25fps progressive or if it has to blend everything to 50fps progressive and then divide in fields or just divide everything in fields 'cause it's already 50fps progressive. Last but not least, it converts everything to yv16 and it applies clipping in order to make sure that Luma and Chroma are in limited tv range (16-235) and then Avisynth gives the resulting uncompressed A/V stream to ffmpeg which encodes it in XDCAM and then remuxes it using BBC BMX Transwrap. Finally, it's gonna deliver the file to a shared storage and it's gonna delete the original file which has already been encoded (if it has been encoded successfully).


By default there are quite a few blocks, but it's very easy to create new blocks and entirely customize them. For instance, this is what my block called "Limiter Tv Range" looks like:

https://i.imgur.com/YRFN3a2.png

Let's zoom in:

https://i.imgur.com/iVgJsQ3.png

As you can see, the user interface is just a visual representation of what Avisynth is doing, therefore it's possible to do pretty much everything you generally do with Avisynth but in automatic and since automatize workflows is something companies like... a lot, I thought that this was the right way forward.
Here's another example; this time, the block is the one in charge of the Loudness Correction:

https://i.imgur.com/evfPhFU.png

See? Each and every block is just like a normal Avisynth Script that you generally write down on a daily basis whenever you have to encode something.
For instance, Limiter Tv Range is basically just clipping:

Limiter(min_luma=16, max_luma=235, min_chroma=16, max_chroma=240)

Loudness Correction is basically just:

ResampleAudio(48000)
Normalize(0.22)
ConvertAudioTo24bit()

Blending and interlacing to 25i is basically just:

ConvertFPS(50)
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()

The channels duplicator for stereo files is basically just:


video=FFVideoSource("file.mov")
audiook=FFAudioSource("file.mov")
audio=MergeChannels(audiook, audiook, audiook, audiook)
AudioDub(video, audio)


and so on...
Oh, just to be precise, I said that it's based on Avisynth, but it's actually based on Avisynth+ (both x86 and x64), so it can use all the additional features of Avisynth+ like regular high bit depth, dealing with different color curves, HDR and so on; for instance, this is what converting a C-Log3 to BT709 linear looks like:

https://i.imgur.com/JQxTvXs.png

Let's zoom in once again:

https://i.imgur.com/qeWfN0x.png

As you can see, it basically takes the input as it is (either 8bit, 10bit, 12bit etc), it brings everything to 16bit planar, then it converts it to RGB, applies one of my LUTs and then it dithers everything down to 8bit with the Floyd-Steinberg error diffusion.

You can basically create many different workflows according to your needs, 'cause the potential of Avisynth is huge; those are some of the watchfolders I made:

https://i.imgur.com/DvrrHi5.png

As you can basically see, it pretty much encodes files for either archival or as mezzanine files to be used later or as files to be delivered to users via whatever service and so on.

Alright, let's crack on and see how encoding looks like, shall we?

https://i.imgur.com/TEVXfVh.png

The monitor looks essential and perfectly productive; you can basically see which workflow is doing what and how long it's gonna take before the encode finishes based on how long a file is and the fps of the encode. It can deliver the resulting file to a folder or to an FTP, according to your needs. In my case, I just set it to deliver the file to a shared folder in a shared storage.
As to the CPU efficiency, it's as efficient as Avisynth and ffmpeg/x262/x264 etc would be on your CPU.
In my case, since I still do many encodes in MPEG-2 and the encoder is definitely not optimized for multithreading, the power of Avisynth comes in parallel encodes rather than single encodes, which can saturate all the cores/threads of my Dual Xeon:

https://i.imgur.com/SziKGJs.png



If you work in a company and need to encode many different files on a daily basis, I strongly recommend you to choose FFAStrans as it's free to use, based on open source programmes like Avisynth+, ffmpeg and so on and it's completely customizable.

This is supposed to be an introduction to this awesome software which I've been using ever since 2016-2017 and that I still use on a daily basis whenever I don't have time to manually write a script and manually encode a masterfile.
Of course, not everything can be done automatically and a manual encode should always be preferred to an automatic one, but let's face it: sometimes having something implemented for you that helps you finishing your work is actually very useful. If you want a complete guide on how to use it with many more screenshots, let me know. In the meantime, here's my personal workflows package which comes pre-loaded with scripts, plugins, and makes use of my LUT collection (https://forum.doom9.org/showthread.php?t=176091):

FranceBB Custom Workflows Package: Link (https://github.com/FranceBB/FranceBB_Sample-Avisynth-Script/tree/master/FFASTrans%20Workflow)

FFAStrans Stable version: Link (https://ffastrans.com/wp/downloads/)

Please note that the legacy 32bit is legacy 'cause it's a very old version with some components modified with XP compatible ones, but you shouldn't really be using it.


Useful Links:

Official FFAStrans website by Steinar Apalnes: https://ffastrans.com/wp/
FFASTrans Wiki: http://www.ffastrans.com/wiki/doku.php

Developer: Steinar Apalnes
Maintainers: emcodem, momocampo, FranceBB

[Gser]

Huh so people really use x262?

[ChaosKing]

The images with the "graphs" are not really readable, could you post a higher res or a version with a zoomed in area?

[FranceBB]

Huh so people really use x262?

MPEG-2 in 2019? Yep. Either x262 or Lavc58.19.102 mpeg2video encoder. Anyway, according to PSNR and SSIM values I've got in many tests their performances are not very good. Still, they are free and open source, so this is what I use. Besides, old playout ports force me to encode in MPEG-2 still today so... this is pretty much it.

The images with the "graphs" are not really readable, could you post a higher res or a version with a zoomed in area?

Sure, I'm gonna zoom them tomorrow and upload a few workflows zoomed in so that they are readable. :)

[FranceBB]

Alright, as requested, here are the zoomed in images.

Let's start with Avisynth_in:

https://i.imgur.com/vU7Bj4u.png

Let's zoom in:

https://i.imgur.com/ZgHv7rs.png

As you can see, once it indexes the audio and video file, HuffYUV is called and it creates a mezzanine lossless video file.
Then, such a video is indexed again and channels are duplicated/ordered accordingly.
Then, Loudness correction to 48'000Hz, 24bit and -24LUFS with Normalize(0.22) is applied.
The next step is yadif which decides whether it's gonna deinterlace it, bob-deinterlace it or leave it as it is if it's progressive.
Of course, this doesn't work with telecined material, which has to be handled differently but once again, as I said before, a manual encode should be preferred over an automatic one and in case of telecined material, you could just make the effort to write

tfm(mode=1,pp=5,slow=2,micmatching=2,clip2=tdeint(mode=2,type=3))
tdecimate()

Anyway, let's keep going with the second zoomed part of the Avisynth_in workflow.

https://i.imgur.com/E493rG8.png

As you can see, blending is applied according to the framerate. The idea is to blend everything to 50fps progressive and then divide into fields upon encoding. If it's over 70fps, then it just drops frames 'till it gets to 50fps. This workflow is mainly used for our news channel, therefore blending is more than enough. For the other channels, I created another conditional which applies speed-up with pitch adjustment if the framerate is 23.976fps, or leave it as it is if it's 25fps progressive and makes a progressive flagged as interlaced.
Then clipping is applied in the "Limiter Tv Range" block and the resulting uncompressed A/V stream is encoded in MPEG-2.
Lastly, the resulting file is remuxed by BBC BMX Transwrap, delivered to a folder and the original file is deleted.
"Pulizia Cache" means "Cache deletion", which basically removes the mezzanine file, the Avisynth Script created and the original file from the temp folder.


Let's take a look at the "Repair_in" workflow:

https://i.imgur.com/qNnzEhl.png

As you can see "Repair_in" is just a fancy name for "Linear Interpolation" performed by mvtools. It basically consists in Indexing, deinterlacing eventually interlaced materials, Resize using Spline64Resize, Linear Interpolation with mvtools, clipping 16-235 limited tv range, encoding and delivering the file, then deleting the temp files/cache.
Sure, linear interpolation ain't perfect and it might introduce artifacts, however it can be useful in some cases like whenever a normal static pan has been recorded by a camera and we want it to be as smooth as possible. MVTools in this case always tries to apply linear interpolation and create a real 50fps progressive file which is then divided into fields to make a real 25i file.

Speaking of MVTools, let's take a look at the workflow "Slowmo_any":

https://i.imgur.com/hkJx1LB.png

Let's zoom in:

https://i.imgur.com/MdpfhC3.png

I gotta say that whenever we need to make a slowmotion, we record it with a very high framerate, like at 100, 200, 300, 500, and even 700fps, however we might not have cameras capable of recording such an high framerate at an high enough resolution; for instance, 700fps can be recorded by our cameras in HD only (1280x720). As a matter of fact, we generally upscale them to FULL HD and 4K. So, whenever we need to slow down those files, we simply use "AssumeFPS(50)".

https://i.imgur.com/Ru3m2bh.png

After AssumeFPS(50), we apply clipping in order to have luma and chroma in Limited Tv Range and then we divide in fields to get a 25i and encode it. If and only if we need to make a slowmotion of something that we didn't record ourselves at high framerate, then we use MVTools and we cross our fingers.

Let's now take a look at this x264 workflow:

https://i.imgur.com/AFU6DpV.png

Let's zoom in:

https://i.imgur.com/2oysEAt.png

As you can see, this workflow is made to produce low-res, low-bitrate proxy files to be viewed by journalists, reporters and so on out of our MPEG-2 50Mbit/s yv16 mezzanine files. In the x264_proxy workflow, the duration of the file is analysed and bitrate is calculated accordingly in order to be as large as 2GB maximum. If it's only few seconds, it would get a too high bitrate, so it's capped at 2500 kbit/s. x264 What'sapp has a similar constrain.

https://i.imgur.com/D9xRiDB.png

x264 and x264 "Archivio Esterne" means "Archive for External Productions" and in this case, x264 can go up to 25 Mbit/s level 4.1, which is a good enough bitrate for a video to be used for productions that are sent on site.

All these workflows make use of custom avisynth scripts and you can create them as well by right-clicking on the UI and either write your own script from scratch, use some of my personalized blocks (Scripts) or use some of the built-in blocks/scripts that Steinar wrote:

https://i.imgur.com/5y783cP.png
https://i.imgur.com/YaMaIBR.png
https://i.imgur.com/cd9NxUJ.png
https://i.imgur.com/TnfIP6r.png
https://i.imgur.com/T4TprpY.png
https://i.imgur.com/rz4GdB1.png
https://i.imgur.com/uQvY2W5.png
https://i.imgur.com/m9gzitf.png

I hope it makes things a bit more clear.
I'll dive into the creation of workflows in a more accurate way if you want.

Cheers,
Frank.

[Emulgator]

Now that's awesome !!!
Deep respect and many thanks !

[Blue_MiSfit]

That's super cool!

Any support for CC / subtitles, or any other embedded VANC?

[FranceBB]

That's super cool!

Any support for CC / subtitles, or any other embedded VANC?

Steinar didn't make any blocks to include subtitles and I didn't create a custom block 'cause I personally use FAB/Cavena to mux .stl and .890 subtitles in .mxf files as other open source alternatives don't really work well with anything other than .srt/.ssa/.ass. Anyway, since FFAStrans is based on Avisynth and FFMpeg, as long as there is a way to handle things with them, you can create a custom block. For instance, I know that with

ffmpeg -f lavfi -i movie=input.ts[out+subcc] -map 0:1 output.srt

you can extract subtitles from a transport stream, however I have no idea whether it's gonna work with an mxf or not.


By the way, I'm gonna re-upload my pre-loaded version to Mega as the link expired.

[mariush]

Isn't HuffYUV kind of slow? I started using MagicYUV years ago and it was much faster and better multithreaded than HuffYUV and there's a free version available: https://www.magicyuv.com/

[FranceBB]

Isn't HuffYUV kind of slow? I started using MagicYUV years ago and it was much faster and better multithreaded than HuffYUV and there's a free version available: https://www.magicyuv.com/

Well, I've been using HuffYUV ever since the very beginning (years ago), however that's something I made myself as a custom block, 'cause in the Steinar implementation (which is the default one) you can use x264 lossless to encode your temporary file which is heavily parallelised. :)

[kolak]

Home version of Telestream Vantage :) (well- not as many features, but priced nicely instead)

[reisub]

Steinar didn't make any blocks to include subtitles and I didn't create a custom block 'cause I personally use FAB/Cavena to mux .stl and .890 subtitles in .mxf files as other open source alternatives don't really work well with anything other than .srt/.ssa/.ass. Anyway, since FFAStrans is based on Avisynth and FFMpeg, as long as there is a way to handle things with them, you can create a custom block. For instance, I know that with

ffmpeg -f lavfi -i movie=input.ts[out+subcc] -map 0:1 output.srt

you can extract subtitles from a transport stream, however I have no idea whether it's gonna work with an mxf or not.


By the way, I'm gonna re-upload my pre-loaded version to Mega as the link expired.
FranceBB, the link is dead. Please upload the "FFAStrans FranceBB pre-loaded version" again.

[FranceBB]

FranceBB, the link is dead. Please upload the "FFAStrans FranceBB pre-loaded version" again.

I wanted to but I'm stuck at home due to the COVID-19. (don't worry, I didn't get it, I'm a healthy young guy xD).
Anyway, everybody but a limited number of people are working from home; they call it "smart working".
They're limiting the access to everybody and I can't get the whole package from the server 'cause my VPN doesn't let me access it directly for security reasons. When this whole mess will end I will, I promise.
As you can figure out yourself, the server I'm talking about with FFASTrans is in Italy.

[reisub]

I wanted to but I'm stuck at home due to the COVID-19. (don't worry, I didn't get it, I'm a healthy young guy xD).
Anyway, everybody but a limited number of people are working from home; they call it "smart working".
They're limiting the access to everybody and I can't get the whole package from the server 'cause my VPN doesn't let me access it directly for security reasons. When this whole mess will end I will, I promise.
As you can figure out yourself, the server I'm talking about with FFASTrans is in Italy.
Ok. Please upload the program when it will be possible for you.

[FranceBB]

Sorry for the delay, but I finally managed to connect remotely to all my servers and after fixing a few issues and making sure everything was running as it should, I made a new backup of the current FranceBB pre-loaded build.
Please note that it's a custom-made dual build of the original Steinar Apalnes implementation, so it's both x86 and x64, but it's supposed to be running on a x64 version of Windows 10.
The fact that it's dual is NOT to make it run on x86 systems, but to make Avisynth x86 plugins run with ffmpeg x86 for backward compatibility reasons whenever it's not possible to have a full x64 workflow.

Processed 1 file, writing output:

Output: "Z:\FFASTrans_v1.0.0.4_FranceBB_preloaded_build.exe"
Install: 2 pages (128 bytes), 1 section (2072 bytes), 12848 instructions (359744 bytes), 84352 strings (807370 bytes), 1 language table (262 bytes).
Datablock optimizer saved 47942 KiB (~16.0%).

Using lzma compression.

EXE header size: 50688 / 37888 bytes
Install code: 129931 / 1170040 bytes
Install data: 256678773 / 1403460488 bytes
CRC (0xA8FF5255): 4 / 4 bytes

Total size: 256859396 / 1404668420 bytes (18.2%)
(source size was 403200292 bytes)

Download: FFASTrans_v1.0.0.4_FranceBB_preloaded_build.exe (https://mega.nz/file/eMtXjC7K#AVCXK8znwpixu9JWPeyfVuE9O6_7DSmXRmoxlwcLwdI)

Note:
Please note that you MUST change the path to make sure that workflows are running properly on your system; remember that this is an ad hoc version made for the company I work for.
Workflows are supposed to be used as an example to make your own workflows.
Several different workflows have been included.
Several matrices of linear transformation have been included.

**from time to time you might find notes written in non-English language as an added bonus xD**

[reisub]

Sorry for the delay, but I finally managed to connect remotely to all my servers and after fixing a few issues and making sure everything was running as it should, I made a new backup of the current FranceBB pre-loaded build.
Please note that it's a custom-made dual build of the original Steinar Apalnes implementation, so it's both x86 and x64, but it's supposed to be running on a x64 version of Windows 10.
The fact that it's dual is NOT to make it run on x86 systems, but to make Avisynth x86 plugins run with ffmpeg x86 for backward compatibility reasons whenever it's not possible to have a full x64 workflow.

Processed 1 file, writing output:

Output: "Z:\FFASTrans_v1.0.0.4_FranceBB_preloaded_build.exe"
Install: 2 pages (128 bytes), 1 section (2072 bytes), 12848 instructions (359744 bytes), 84352 strings (807370 bytes), 1 language table (262 bytes).
Datablock optimizer saved 47942 KiB (~16.0%).

Using lzma compression.

EXE header size: 50688 / 37888 bytes
Install code: 129931 / 1170040 bytes
Install data: 256678773 / 1403460488 bytes
CRC (0xA8FF5255): 4 / 4 bytes

Total size: 256859396 / 1404668420 bytes (18.2%)
(source size was 403200292 bytes)

Download: FFASTrans_v1.0.0.4_FranceBB_preloaded_build.exe (https://mega.nz/file/eMtXjC7K#AVCXK8znwpixu9JWPeyfVuE9O6_7DSmXRmoxlwcLwdI)

Note:
Please note that you MUST change the path to make sure that workflows are running properly on your system; remember that this is an ad hoc version made for the company I work for.
Workflows are supposed to be used as an example to make your own workflows.
Several different workflows have been included.
Several matrices of linear transformation have been included.

**from time to time you might find notes written in non-English language as an added bonus xD**

Thanks for the build. What is the password to change workflow?

[FranceBB]

Thanks for the build. What is the password to change workflow?

Modified without password. ;)

[FranceBB]

Updated build to v1.0.0.5

Changes:

- Improved conditional logic in some workflows.
- Ability to re-create the exact folder structure in x262_in and Avisynth_in
- Introduced x265 lossless mode
- Introduced an anti-aliasing block
- Introduced a post-processing block for old BetaCAM sources with NNEDI3 + Spline64 upscale, automatic white balancing, automatic luma/chroma adjustments and basic sharpening.
- General code cleanup in customized workflows
- Temporarily removed FrostyBorders

New Build Download (https://mega.nz/file/jQdw0axb#77Ot8wY11BktrtEE4MK9gRCtJHo0NdMkMx4PkcnFBWc)

This is a new updated build which contains some of the workflows I made that are meant to be used in the broadcast company I work for.
Unfortunately, some of them have not been included, but the reason is that those I removed were using FTP or had other critical credentials to access our internal server inside our .bat or other blocks and therefore I couldn't safely share them.
Anyway, those inside the build are pretty much the most used one, so they should cover a broad range of cases.
They're mainly focused on MPEG-2 XDCAM50 encoding, however, they can be easily modified and used to encode and deliver other formats.
The only "catch" is that they're really meant for PAL regions, so I don't know how useful can be for those living in NTSC countries.
Please note that we have three platforms:
TG24 which stands for "Television Journal 24h" in other words a round-the-clock news channel. It's basically the non-English version of Sky News.
SPORT which is basically dedicated to sports channels.
PCE which stands for "Production Cinema Entertainment" which is dedicated to our cinema channels, our original productions, our general purpose channels and so on, but only for "promo" i.e "commercial ads", in fact the real programs that fall in this category are encoded with manually written scripts and 16bit planar precision post-processing to deliver the best possible quality all the time and those tasks couldn't be automatized (why would I even care to go to work if they could? xD).
Anyway, those three platforms have three different RAID-6 storages in which there are different input folders, namely "Avisynth Server 1" -> TG24 / SPORT / PCE -> "Node 1 / 2 / 3 / 4" -> OUTPUT.
There's a conditional logic that outputs the file to the correct folder according to which platform is being used (i.e in which server the input folder is) and which node has been used.
There are also some workflows that deliver to either Vantage or to MediaDirector so that they can check-in the file inside AVID Interplay Access automatically.
Alright, let's get started and dig into our workflows.

01 - AppleProRes_in

Image: Link (https://i.imgur.com/bk8n0Ed.png)

This is a very simple workflow.
It's not really used much as it's just a quick and dirt AppleProRes encode using iCodecPro from ffmpeg.
The only reason it's there is that sometimes, when we have to deliver files (that have already been post-processed and encoded and that are ready to air) to other companies they ask us to send them in AppleProRes, so I needed something to automatize it.

02 - Avisynth_in & ANY

Image: Link (https://i.imgur.com/MTkAEbd.png)

This is more like it.
This workflow is one of the most used.
It basically uses Avisynth x64 to index the source file, it checks the audio channels and make sure to always have 8 mono channels (if you have like a stereo file with just FL FR, it then maps CH1 FL CH2 FR, CH3 FL CH4 FR CH5 FL CH6 FR CH7 FL CH8 FR) and it performs a Loudness Correction to -24 LUFS, then it bob-deinterlaces everything if needed, and it decides what to do:
if it's 23.976fps or 24fps it performs a 4% speed-up + pitch adjustment to 25p and encodes it as progressive flagged as interlaced
if it's 25p it leaves it alone and encodes it as progressive flagged as interlaced
if it's 25i it bob-deinterlaces it to 50p and then re-interlaces it correctly to avoid any interlacing artifacts
if it's below 23p then it blends it to 50p, divides it in fields and encodes it to 25i
if it's above 24p but below 50 it blends it to 50p, divides it in fields and encodes it to 25i
if it's 29.970p then it blends it to 50p, divides it in fields and encodes it to 25i
if it's 29.970i then it bob-deinterlaces it to 60p, blends it to 50p, divides it in fields and encodes it to 25i
if it's 50p then it divides it in fields and encodes it to 25i
if it's 60p then it blends it to 50p, divides it in fields and encodes it to 25i
if it's above 60p but below 70p, it blends to 50p, divides it in fields and encodes it to 25i
if it's above 70p then it just drops frames to 50p, divides it in fields and encodes it to 25i

Then it brings luma and chroma in Limited TV Range 8bit planar 0.7mV 16-235 by clipping, lastly it encodes and delivers the file.

Of course, if you have a structure like:

Folder: Shooting\Day1\CameraA\000454.mov

it creates the very same path in the output:

Folder: Shooting\Day1\CameraA\000454.mxf

so if you have several folders and sub-folders with many different files inside, you can copy all of them in the input watchfolder and it will create them back in the output.

Lastly it will perform "Cache Cleanup" just like any other workflow which means that the input file is gonna be deleted along with all the temporary files created in-between.

03 - Avisynth_in Antialiasing SLOWMO

Image: Link (https://i.imgur.com/MuJ25bt.png)

It performs a slow-motion of the input files using AssumeFPS, then it performs anti-aliasing and brings luma and chroma to limited TV range and encodes.
Audio is preserved and slowed down as well when possible (i.e unless the clip is slowed down too much).


04 - Avisynth_in P2

Image: Link (https://i.imgur.com/nGoSMNk.png)

This takes a Panasonic P2 folder structure which has 1 video and 4 different mono PCM audio channels NOT Embedded (i.e not muxed together).
It uses the built-in function of FFASTrans which reads the .xml metadata and uses them to detect video and audio files and put them together using Avisynth.
After that the very same logic is applied, namely Loudness Correction to -24 LUFS and Luma and Chroma in Limited TV Range.
The user can choose whether the files within the folder have to be appended together to make a single final Masterclip or whether they should be encoded one at a time.

05 - Avisynth_in SD

Image: Link (https://i.imgur.com/IkONRa2.png)

This is rarely used nowadays, but it takes the file in input and encodes in IMX50.
The same logic is applied: 8 audio channels, Limiter TV Range and Loudness Correction.

06 - Avisynth Remaster

Image: Link (https://i.imgur.com/UutUaLM.png)

This is still at a very beta stage and it hasn't been used yet other than for some tests.
Basically, what it does, is that it receives an old SD file and it upscales it to FULL HD in the least painful way (NNEDI3 + Spline64 with 16bit planar precision) and it also performs automatic white-balancing and it attempts to fix color-dominance if detected (i.e blue tint etc). Lastly, it performs a quick and dirt sharpening, clips everything to 16-235 and performs loudness correction.
It's still a beta 'cause I still have to decide which Sharpening filter I'm gonna use (LSFMod, SeeSaw etc) and whether I'm gonna use GamMac in a sort of automatic fashion or not.
Besides, I still haven't made up my mind about which option I'm gonna use to go from BT601 to BT709: the old ColorMatrix which works in 8bit planar only? Matrix from HDRTools which works in 16bit interleaved? I still have to make a few tests.
Last but not least, I don't know whether I'm gonna go on with it or if I'm gonna include the whole Fred Van de Putte series of filters instead.
Again, I gotta test.

07 - DNxHD

Image: Link (https://i.imgur.com/zYuVWa7.png)

Quick and dirt workflow that does exactly what the AppleProRes one does, but for DNxHD.

08 - HuffYUV_in

Image: Link (https://i.imgur.com/4kuFeSQ.png)

This is for lossless mezzanine file only that have to be encoded as it encodes the input in HuffYUV lossless.

09 - Linear Interpolation

Image: Link (https://i.imgur.com/mM1QVwP.png)

The workflow speaks for itself as it uses MVTools and its forward and backward vectors to perform linear interpolation to 50p in case it's needed.
This workflow does not maintain AR (contrary to the other workflows) and it's only used when it's really necessary, like if we have a mixed content show and we want everything to be at true 25i and smooth, however it's only used if the director specifically tells us to do so as he becomes responsible for eventual artifacts created every now and then by linear interpolation and of course if there are too many or if he thinks that it's not the result he wanted, he can definitely tell us to do it again without linear interpolation and have an hybrid show instead (progressive + interlaced).

10 - Linear Transformations 1, 2, 3

Image: Link 1 (https://i.imgur.com/CScJUHR.png) - Link 2 (https://i.imgur.com/NwjszQl.png) - Link 3 (https://i.imgur.com/DX0TixY.png)

These 3 workflows apply one of my matrices of linear transformation (https://forum.doom9.org/showthread.php?t=176091) to map each and every point of a curve into another with 16bit precision.
All calculations are done in PlanarRGB 16bit and the result is dithered down to 8bit planar using the Floyd-Steinberg error diffusion.
The resulting output is then limited to 8bit TV Range 16-235, loudness corrected to -24 LUFS and encoded.
It performs the following linear transformation:

HLG BT2020 -> Linear BT709
PQ BT2100 -> Linear BT709
S-Log3 -> Linear BT709
C-Log3 -> Linear BT709
Log-C -> Linear BT709
D-Log -> Linear BT709

Of course it can be replicated and used with any of my LUTs to perform whatever linear transformation you want.
As for HLG and PQ, I've been thinking for a while about using a tone-mapping algorithm instead, namely Reinhard, using the plugin from Jean-Philippe Scotto di Rinaldi (although the one from Donald Graft is also a perfectly valid alternative, so I don't which one I'm gonna use or if I'm gonna keep a custom LUT instead).

11 - Slowmo ANY

Image: Link (https://i.imgur.com/CyhdI7D.png)

It performs slow-motion using AssumeFPS() to anything unless it's 25p. In that case it leaves it as it is.
It's the same as Slowmo-Antialiasing, but it doesn't do any antialiasing.
As always, the output is brought to 8bit Limited TV Range 16-235.

12 - x262_in

Link: Image (https://i.imgur.com/m5HDNnV.png)

It's a conversion to XDCAM 25i without using any fancy conditional logic or Avisynth.
It started with x262, but I then changed it to ffmbc and now it's using the MPEG-2 encoder from ffmpeg with the built-in logic of the XDCAM block made by Steinar Apalnes.

13 - x262 DVD in

Link: Image (https://i.imgur.com/1OjoEpB.png)

Just like the one above, it started with x262 but now it's using the MPEG-2 encoder inside ffmpeg and it produces an ISO file that it's ready to be burned in a PAL DVD.
The reason why there's no manually-written conditional logic at all other than the one written by Steinar for this particular block is that if we make a DVD it's from one of our final file that is already ready to go on air, therefore nothing fancy has to be done other than interlace-aware downscaling and BT709 to BT601 conversion.
As a matter of fact, this replaced the old method of using physical DVD recorders to literally make a DVD from the hardware-downscaled FULL HD SDI input.

14 - x264

Link: Image (https://i.imgur.com/Vrs8uex.png)

First of all, we don't internally encode to H.264 straight away and it's generally used as something that we send to either users or external productions that need it as a backup.
There are quite a few watchfolders, some of them encode in FULL HD with a very low bitrate, some other downscale to a ridiculous resolution and encode it with a file-size that is just right for WhatsApp, some other make a crappy proxy that is used... you know... as a proxy to just know what that file is about and one is used to encode the input files to 50p FULL HD BT709 4:2:2 8bit planar 25 Mbit/s and that's the file that is sent as a very last resort backup to external on-field productions if everything else fails.
For instance, if they're in an OBI-Van and the masterfile of the intros can't be used and their second HDD backup fails and they can't connect to our storage and their disk backup fails as well then they're gonna use this 25Mbit/s H.264 file for the opening before the program begins. Opening means just a small intro or a logo, a bumper etc.


15 - x265

Link: Image (https://i.imgur.com/3QtWQY9.png)

It can be used to either encode a file that is already ready to go but in H.265 with a very low crf to be delivered to users via internet (nothing linear) or as an alternative to HuffYUV for lossless mezzanine file if the x265_lossless fodler is selected instead.

16 - Avisynth Metadata

Link: Image (https://i.imgur.com/ep7cJ3c.png)

The name is misleading since it doesn't have to do anything with metadata, but it's the name of the department who uses it "Metadata Management".
Basically those are cheap low-res proxy that are generally sent away to celebrities that have been interviewed and wanna have a copy before it goes on air.
It's 25p, low quality, H.264, with a logo and a timecode to avoid leaks since we don't really have any control of the file once it has been sent out to the celebrity.
As a matter of fact, it's done by another department and not the one I work in 'cause they are aware of legal rights and other stuff about what can and cannot be done (I don't know any of that so I don't really mess up with that).


Alright, that's pretty much it. :)

Have a good week-end,
bye!

[FranceBB]

Every time I get my hands on a workflow to fulfill a new requirement, I get the feeling that things are getting a bit out of hands (i.e they're getting enormous xD)

https://i.imgur.com/tr0Q9mI.png

[FranceBB]

Updated workflows and moved them to Github: https://github.com/FranceBB/FranceBB_Sample-Avisynth-Script/tree/master/FFASTrans%20Workflow
See first post.

[kolak]

Every time I get my hands on a workflow to fulfill a new requirement, I get the feeling that things are getting a bit out of hands (i.e they're getting enormous xD)



Maybe introduce "Forward/Receive" action which is what Vantage does.
Then 1 big workflow can be split into few smaller ones and output of each gets passed to next one. Easier to organise and manage.

[FranceBB]

I was writing a Wikipedia Page about FFASTrans but I'm struggling to get it approved 'cause they say that it lacks official references...

Here's the page: https://en.wikipedia.org/wiki/Draft:FFAStrans_-_FFMpeg_Avisynth_Transcoder#FFAStrans_-_FFMpeg_Avisynth_Transcoder

Any suggestions? Have you ever managed to get something approved on Wikipedia?

[patul]

He was complaining about COI, conflict of interest, as if you "advertise" the software. I don't know how to resolve it though, probably a further read on this page will help, https://en.wikipedia.org/wiki/Wikipedia:Conflict_of_interest. A suggestion on writing the content, probably you need to cite the references within the content, rather than just put up reference list.

[FranceBB]

He was complaining about COI, conflict of interest, as if you "advertise" the software. I don't know how to resolve it though, probably a further read on this page will help, https://en.wikipedia.org/wiki/Wikipedia:Conflict_of_interest. A suggestion on writing the content, probably you need to cite the references within the content, rather than just put up reference list.

The COI wasn't originally there, it came after I wrote him a message saying him that I wanted to talk to him to discuss what I could improve and modify to get it approved, but rather than replying to me via message, he flagged it as COI... Not the reaction I expected, given that I wanted to ask the reviewer what I could do better to improve the page and get it accepted... :(
Anyway, about the references, I tried to add as many as I could, but it still didn't work... :(

Maybe introduce "Forward/Receive" action which is what Vantage does.
Then 1 big workflow can be split into few smaller ones and output of each gets passed to next one. Easier to organise and manage.

Well, I decided to split my workflow and it worked.
The way I did it was by calling other workflows using the REST-API, so what you suggested is exactly what I did.
The way it's implemented right now, though, requires a bit of knowledge and scripting of the REST-API, but it can be done. :D

[patul]

I meant for inline citation, not just reference. Please see in the picture below. I'm not really good in English, sorry.

https://i.ibb.co/XkG6BnZ/RCapture.png

[FranceBB]

Ah!! I see!
Yeah, it makes sense. I'll try adding that. :)

[FranceBB]

Version 1.2.0 Stable Released!! (https://github.com/steipal/FFAStrans-Public/releases/download/1.2.0/FFAStrans1.2.0.7z)

Changelog:

- New filter: Safe Color Limiter makes sure you video stays within legal TV ranges.

- New filter: Linear Transformation for linear conversion between different color formats like HDR <-> SDR.

- Both conditional and populate nodes now supports dynamic number of fields and resizes the GUI accordingly.

- Added new operator mode to support regular expressions in the conditional node.

- Added the ability to send attachments and a "Reply to" field to the "Send E-mail" node.

- Added HD profiles to the "DV/DVCPRO" encoder.

- Added lasso functionality for selecting several nodes in the editor.

- Added new function $regreplace() for replacing strings using PCRE regex patterns.

- Added new function $jsonput() for adding JSON data to an existing or create new JSON formatted string.

- Added new variable %s_wf_folder% for getting folder of the running workflow.

- Added new variable %i_job_priority% for getting priority of the running job instance.

- Added new variable %s_split_id% for getting split id of the running job instance.

- Added file count to the "Hold" node wait for file option.

- Added option in "Custom AviSynth" filter to attempt adapting colors pace metadata to video properties.

- Copied nodes with connection within the group is now kept.

- Selected nodes can now be moved simultaneously while keeping their relative position.

- The "YouTube"-decoder will no longer try to download complete lists.

- Improved start TC detection on some formats.

- The "H.264" encoder can now produce MXF files and uncompressed PCM audio in the MOV, MKV and MXF wrappers.

- The "A/V Media" decoder now dither if source video bit depth is higher than specified in the node.

- Increased the number of max active jobs the inbuilt monitor would display.

- The rest service now returns 503 if database is not available. Mostly a problem when running FFAStrans from share.

- Aborting jobs will now try more graceful shutdown of child processes before force killing.

- General improvements to file monitoring and job initiation reliability.

- Addressed several stability issues when running FFAStrans from a share.

- Fixed issue where running FFAStrans would not properly close on shutdown.

- Submitting files to other workflows than the current in editor did not work.

- Video with too short audio could result in corrupt endings.

- Results from PSNR and SSIM in "Video quality" analyzer was switched.

- Fixed issues with catching all STDOUT data from "Command Executor".

- Fixed GUI issues when controlling manager through TeamViewer.

- Fixed several reliability issues with the camera monitors.

- Fixed crash when attempting to use predefined levels in "Colorspace" filter.

- The workflow counter in FFAStrans Workflow Manager lower status bar did not work.

- Fixed issue with the FFAStrans Workflow Manager causing graphic issues and instability when working with large workflows.

- Custom nodes would crash if trying to open a help section that did not exist.

- Variables within variables was not properly expanded.

- The web interface would sometimes show hanging node even though it had finished.

- When submitting file through the API, the job id was initially written to a wrong key.

- Fixed crash that sometimes occurred when enumerating child processes.

- Variables or statics with data containing the pipe "|" character was not displayed in the variables list.

- Numerous other minor bug fixes and improvements.

Download: Link (https://github.com/steipal/FFAStrans-Public/releases/)

The website has now been re-written from scratch: http://ffastrans.com/wp/

Cheers,
Frank

[FranceBB]

Version 1.3.0 Stable Released!! (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.0/FFAStrans.1.3.7z)

Finally a new version that doesn't just focus on fixes and improvements but that also has a lot of new nice features.

Changelog:


New workflow features:
- Sub splits: This feature enables sub connected splits (brown) to always finish before the main splits (black).
- Workflow connection: Enables you to forward a job to another workflow.

New functions:
- $lookup() and $lookuprep() for table lookup and lookup replacement.
- $sort() for sorting an array or a string.
- $split() for splitting a string into an array.
- $count() for counting occurrences of a string or single character.
- $foreach() for dynamically executing operations on an array
- $string() for formatting strings like printf in C.
- $xxhash64() for 64 bit version of xxhash.

New node:
- “For each”: Will loop through an array of values and dynamically create a new split/branch for each index.


New variables:
- %s_start_tc_alt% for alternative start TC taken from creation date of media.
- %s_gamma% for gamma curve of media
- %s_chroma_loc% for chroma location of media.
- %i_gmt_bias% for the local GMT bias.
- %s_version% for current running FFAStrans version.
- %s_last_error% for retrieving last viable error message.

New API methods:
- GET /workflows/status for getting current status
- POST /workflows for importing FFAStrans exported workflows.
- POST /presets for importing FFAStrans exported presets.
- POST /user_variables for importing FFAStrans exported user variables.
- GET /history now supports ?start and ?count for getting custom range.

Others:
- All encoders now support applying custom or inbuilt LUTs.
- All encoders now support setting and converting full/limited video range.
- Huge speed improvement when muxing to BMX MXF as FFAStrans no longer creates an intermediate file before remuxing.
- Added search function in the variables selector.
- Added free text search function for workflows and nodes.
- The XAVC node now support class 300 and 480 in UHD.
- The AVC-Intra node now support class 2K and 4K.
- Added option to use user variables as framerate in “ProRes” and “DNxHR” encoder nodes.
- Added user selectable quality tunings to the the “XDCAM-HD” node.
- Added “Writing library” mode to the “ProRes” encoder node for improved compatibility in some scenarios.
- Added HLG and PQ signaling to the “H.264” encoder.
- Added option to save complete VMAF json to a variable in the “Video Quality” node.
- Added access to the user variables from main manager menu.
- Added user selectable zero padding to unique output file names in the delivery nodes.
- The header section in “HTTP communicate” node now supports using variables.
- The “%s_original_path~n%” variable now supports minus notation for retrieving path names from the right side.
- The “%s_recursed_path%” variable now support the same notation type as %s_original_path~n% with %s_recursed_path~n%.
- Added option to export job logs from inbuilt status monitor.
- The “Hold” node now supports arrays as list of files.
- Updated to use the new maintained yt-dlp instead of youtube-dl.
- Changing a variables/static name now automatically updates all applicable workflows.
- Expanded the $exists() function to support path recursing and returning a file list in array.
- Added support for adding data to an existing array using the $jsonput() function.
- Fixed issue where the “H.264” encoder produced corrupted MXF files.
- Fixed various bit depth issues in the “H.264/5” encoders.
- Fixed “Generic MPEG” was forced to 16:9 aspect.
- Lots of minor changes and improvements.


Download: Link (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.0/FFAStrans.1.3.7z)

[FranceBB]

Version 1.3.0.1 Stable Released!! (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.0.1/FFAStrans1.3.0.1.7z)

Changelog:


- Fixed potential endless loop in “A/V Decoder” node.
- Fixed a behaviour that would make the XDCAM-HD encoder would sometimes stop encoding on very complex content.
- Fixed an issue where sources with yuvj420p pixel format would sometimes lead to failed encoding.
- Fixed an issue that caused bad frame rate conversion on small fps difference between source and output.
- Fixed an issue where host settings was not properly saved.
- Fixed error message on multiple immediate retries from failed node.


Download: Link (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.0.1/FFAStrans1.3.0.1.7z)

[FranceBB]

Version 1.3.0.2 Stable Released!! (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.0.2/FFAStrans1.3.0.2.7z)

Changelog:


- Fixed progressive closed GOP encoding in XDCAM-HD.
- Fixed a bug in the “Video quality” node which would sometimes return a bogus “inf000000” value.
- Fixed a bug in which the “Video quality” node would fail if source or ref was IMX-D10 content.
- Fixed an issue which caused the “%s_cache_record%” variable to not be correctly populated.
- Fixed the “Sequence” monitor disregarding the “Recurse” setting.
- Fixed a bug in which the “Sequence” monitor would sometimes not work due to an invalid input option being set.
- Fixed a bug which caused two succeeding encodings nodes on the same branch to fail when using the inbuilt encoders.
- Fixed minor issues with properly recognizing some pixel and color formats when processing.
- Fixed an issue which caused the loudness table in “Audio extract” encoder gui to not set a value.
- A few other minor edge case issues addressed.



Download: Link (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.0.2/FFAStrans1.3.0.2.7z)

[FranceBB]

Version 1.3.1 Stable Released!! (https://github.com/steipal/FFAStrans-Public/releases/download/1.3.1/FFAStrans1.3.1.7z)

Changelog:


- Added 4:4:4 profile (XQ) to the ProResKS encoder.
- Added "MXF (BMX)" wrapper option to the Panasonic P2 monitor.
- Added option for removing farming settings when exporting workflows
- Added PUT method for updating existing workflow through API
- Added user option to read from STDERR and/or STDOUT in the "Command executor" node.
- Improved general robustness when executing many jobs at the same time.
- Reduced general priority on sub processes in order to not halt system.
- Fixed some special case FTP monitor issue that caused files to not be downloaded properly.
- Fixed framerate bug when encoding to BMX (MXF)
- Fixed an issue where setting %s_source% to "" (nothing) resulted in some erratic behavior.
- Fixed some job retry issues.
- Fixed audio config crash in the DV encoder.
- Fixed rare memory leak issue.
- Fixed MOV bug in the DNxHD encoder
- Fixed a problem with the A/V Media decoder created zero-length clips.
- Fixed an issue that caused some image sequences to not be properly parsed
- Fixed a problem where plugin processors GUI got stuck behind manager window.
- Fixed a bug where the "Fade" filter ignored audio/video enable/disable
- Fixed an issue that caused the "Custom FFmpeg" encoder to not work properly when using custom ffmpeg mapping.
- Fixed an API issue where history was not always correctly returned.
- Fixed an issue causing some AviSynth based video filters to not work properly on older CPUs
- Fixed a flaw with priority connections in conjunction with executing on error.
- Several other minor improvements and fixes

[FranceBB]

FFAStrans 1.4.0.1 Released!! (https://github.com/steipal/FFAStrans-Public/releases/download/1.4.0.1/FFAStrans1.4.0.1.7z)

Changelog:


- Hosts group: Group hosts into different named groups and assign nodes or/and workflows to them.

- Job slots per node: Assign a higher number of job slots to a node in order to better "allocate" resources on the executing host.

- Cron scheduling support for monitors

New functions:
$dateweek(): Calculate week numbers
$timecalc(): Time calculator
$waccess(): For testing write access
$owner(): For getting owner of a file
$asplit(): Split/format array into a string
$ffconcat(): For creating a "ffconcat version 1.0" string
$shortcut(): For parsing Windows shortcuts and macOS alias files

- New variables:
%i_v_precharge%: Get precharge value from MXF files
%s_options%: System variable for advanced altering of job behavior
%f_pix_aspect%: Pixel aspect ratio as float
%s_frame_rate%: Represent frame rate as num/denum string

- New API method:
GET /jobvars/ for getting final value of user variables after job end

- Improved processing speed, quality and color precision for all encoders.
- Huge database I/O optimizations for those with larger farms.
- Dolby E decoding in the "A/V Decoder".
- "Forget missing files" are now supported by all applicable monitor nodes.
- Support the new frame properties in AviSynth+.
- Support Mac alias in the "Folder" monitor and improved Windows shortcut support.
- Improved support for encoding files with cover art.
- Properly support audio buffer files > 4GB in the "A/V Decoder".
- Support reading stdout/err in command executor as unicode.
- Support for XDCAM-EX 35 720p in the "XDCAM-HD" encoder.
- Support multiple selection in the status monitor.
- Inbuilt status monitor now shows history pages instead of one huge list.
- Improved "Color conversion" filter.
- Improved navigation in the status monitor using arrow keys.
- Improved up-time reliability.
- Native support for "ffconcat version 1.0" input sources.
- Native support for reading IMF packages.
- Support writing 64 bit audio files in the "Extract audio" node.
- Support UTF-8 formatted mail bodies in "Send E-Mail" node.
- Playlist support in "YouTube" decoder and save list to variable.
- Clear complete work cache dir from FFAStrans Manager.
- Support SinPowResize kernel in "Resize" filter.
- Improved automatic cleaning of log and work files.
- FFAStrans will now archive job logs and keep them for an extended period.
- Fixed several long path issues in monitors and FTP delivery.
- Fixed slow opening of long workflows.
- Fixed comparing negative number not working in the "Conditional" node.
- Fixed graphical artifact when moving too far out on a large workflow.
- Fixed some number like strings being strictly regarded as number.
- Lots other minor fixes and improvements.


https://i.imgur.com/X2PfVwz.png

[FranceBB]

FFAStrans 1.4.0.5 Released!! (https://github.com/steipal/FFAStrans-Public/releases)

Changelog:


- Fixed SAR was not set when auto cropping IMX source in the A/V Decoder
- Fixed Interlace analyze node reported bogus results
- Fixed wrong progress on video analyze node
- Fixed rare issue where priority branches did not end properly
- Fixed issue that in some conditions caused a job queue to build up
- Fixed A/V Decoder not properly decoding some broken MPEG2
- Removed invalid 4:4:4 AVI option in the Uncompressed encoder node
- Removed too strict variable check when submitting jobs via API
- Fixed continuously growing files being picked up too early
- Fixed multiple hosts picking up the same growing file
- Fixed loudness analyser not returning correct channel layout
- Fixed error in priority splits
- Fixed stills decoder having malformated color conversions.
- Fixed rare crash on badly formated time code
- Fixed a file locking anomalies


https://i.imgur.com/iWGQFYZ.png

[FranceBB]

Well, those were from 4 years ago and although they have been working fine up until a while ago, they're not gonna work with 1.4.x 'cause starting from 1.4 we have the Avisynth frame properties.
Before frame properties either you had an input, you submitted the file to an encoder node and let the filter_builder take care of creating a command line for you to encode the file properly by performing all the necessary conversions like deinterlacing, converting the primaries, transfer and matrix etc or you could just write your own AVS Script and then basically tell the filter_builder "hey, don't do anything, I've done everything, just encode the file".
In other words, there was no in between scenario.
Now, with the new frame properties, we actually have a way to interact with the filter_builder.a3x which means that we can have scenarios in which you're doing some things within your Avisynth Script but you don't wanna take care of some other things and you want the filter_builder to do it for you. This is where frame properties come in handy.
I'll update the workflow collection on Monday, first thing in the morning. You're gonna be able to see where frame properties are used and how. :)

[FranceBB]

FFAStrans 1.4.0.6 Released!! (https://github.com/steipal/FFAStrans-Public/releases)

Changelog:

- Fixed some illegal characters when parsing mac aliases.
- Fixed occasional double registration of new monitored files.
- Fixed loudness analyser not storing correct real peaks data.
- Fixed rare error that sometimes caused retried jobs to not start.
- Fixed audio files getting double length when converting to video formats.
- Updated BMX to include fix for wrongly flagged aspect ratio on some MPEG-2 SD formats.
- Fixed rare bug that sometimes caused FFAStrans to not continue with jobs containing priority splits.

[FranceBB]

FFAStrans 1.4.0.7 Released!! (https://github.com/steipal/FFAStrans-Public/releases)

Changelog:


- Fixed a bug occurring when monitoring growing files.
- Fixed a stills encoder bug occurring when encoding a single image.
- Fixed a potential race condition bug occurring when several nodes are parsing the same media source.
- Fixed a rare API bug occurring when submitting huge JSON bodies.
- Fixed a color scheme bug in the workflow manager affecting the representation of bypassed nodes.
- Fixed a bug that caused all running workflows to restart when just one workflow was being stopped.
- Fixed a bug that could cause FFAStrans to not clean up old jobs/logs properly.
- Fixed some aggressive enumeration of history records.
- Several other minor bug fixes and improvements.

[anton_foy]

Awesome! I like to start to use it but I have not found if (probably the answer is no) it supports encoding in >10-bit for dnxhr/prores or if you can recommend another codec for the use in post production? Thanks!

[FranceBB]

Unfortunately you're right, but there's nothing I can do about it. The problem with DNX and ProRes is that both implementations rely on the Lavc encoder which is built in ffmpeg and nobody ever developed it to make it go past 10bit planar, so we can't just add an encoding profile that doesn't exist. :(
If someone is gonna develop a new high bit depth version of the DNX HR and AppleProRes encoder that supports 12bit, we'll definitely add support for it, but until then I'm afraid we don't have much choice. :(

[anton_foy]

Unfortunately you're right, but there's nothing I can do about it. The problem with DNX and ProRes is that both implementations rely on the Lavc encoder which is built in ffmpeg and nobody ever developed it to make it go past 10bit planar, so we can't just add an encoding profile that doesn't exist. :(
If someone is gonna develop a new high bit depth version of the DNX HR and AppleProRes encoder that supports 12bit, we'll definitely add support for it, but until then I'm afraid we don't have much choice. :(

I understand thanks. But in virtualdub2 I can go 16-bit with some lossless codecs. A tad too big files when in 4k. If not dnxhr or prores then any other codec that can do >10-bit in ffastrans?

[FranceBB]

I understand thanks. But in virtualdub2 I can go 16-bit with some lossless codecs. A tad too big files when in 4k. If not dnxhr or prores then any other codec that can do >10-bit in ffastrans?

Yeah when using HuffYUV and UTVideo files can get really big really fast. Currently there isn't a native encoding node for those, but I can add it to the lossless encoder which is currently v210 only. Of course one could just use the custom ffmpeg script and encode in any codec supported by ffmpeg like ffvhuff in this case, however that would skip the filter_builder entirely. Basically when an encoding node is used the filter_builder creates the right filter chain for you to prepare the input stream and make it ready to be encoded in whatever target you want. The filter_builder.au3 tackles each "problem" one step at a time, so for instance if your source is an Slog3 and your target is BT709, it knows that it has to go to RGB 16bit full range first, apply the appropriate LUT and go back to YUV. If your source is UHD and your target is FULL HD it knows that it has to downscale (it allows you to crop or add borders or stretch) and so on and so forth. It knows about interlacing, about chroma placement, about matrix, transfer and primaries etc. it even knows that when going from a progressive input at an higher resolution to an interlaced output at a lower resolution (let's say from UHD 50p to FULL HD 25i while dividing in fields) it needs to preblur vertically to avoid aliasing. I mean, the filter_builder is there to help but there's a big "but": when using a custom ffmpeg script (i.e encoder) it won't do anything as it doesn't know what you wanna do, which means that if you use ffvhuff in a custom ffmpeg script you'll have to tackle everything yourself. Of course, you can still use an indexer (i.e an A/V Decoder node) so that you can write your own logic in Avisynth and prepare the input stream to be encoded based on a bunch of conditionals and/or custom Avisynth scripts. In other words, if you need to encode in FULL HD 4:2:2 Slog3 25p 12bit you can output exactly that from Avisynth so that the encoding node won't have to do anything weird and it will only have to encode. That being said, you can also mix and match so that if you use one of the encoding nodes but you don't want the filter_builder to take care of everything, you can still do that in Avisynth provided that you then set the appropriate frame properties via PropSet() so that the filter_builder is gonna know that you already took care of that yourself. Speaking of encoders with high bit depth support (higher than 10bits) the only one that pops to my mind is the H.265 encoding node which is obviously calling x265, so it supports 12bit out of the box, but if you're interested in things like HuffYUV and UTVideo you're gonna have to go with a custom ffmpeg script inside FFAStrans I'm afraid. But please let me know and in case I'll add them to the lossless encoder. :)

[anton_foy]

Cool I like to try out the h265 12-bit option first. Now Im very newbie to encoding options beside the prores / dnxhr. For post production I-frame and other settings to make it comparable to prores/dnxhr is something I would like to adjust. Preferably better if possible but not fully lossless.

[FranceBB]

FFAStrans 1.4.1 Released!! (https://github.com/steipal/FFAStrans-Public/releases)

Changelog:


- VMAF now uses all cores available (major speedup).
- Removed all 8-bit limitations from all applicable filter nodes.
- The "Insert Media" and "Overlay" filters now support any type of video format.
- "Insert Media" filter now supports "Overlay" mode for media supporting alpha channels.
- The "Stills" encoder now also support outputting actual source frame numbers.
- Generally improved handling of "odd" formats.
- Added periodic sampling option to "Video Quality".
- Added start, repeating and stop scripts/program per workflow.
- Support Avid Source Path/File columns by inserting "file://" prefix in Tape (.ref) field.
- Changing priority on running jobs can now be done from the inbuilt status monitor.
- Added custom work dir in the "Command executor" node.
- Added bigger popup input editor for long text in input fields.
- Added tooltip for previewing long text in input fields.
- Added %f_capture_rate% variable for reading camera capture frame rate.
- Improved error feedback from YouTube node.
- Improved parsing of correct media length.
- History logs are now always backed up when resetting status monitor.
- Fixed Rest API that would in some rare cases not return a proper reply.
- Addressed issues with jobs seemingly being stuck in status monitor.
- Fixed rare cases of broken interlaced processing.
- Fixed failed encoding from unknown audio codecs.
- Fixed not checking default value for force leading zeros on dates.
- Fixed broken encoding error status feedback.
- Fixed some rare cases of timecode not being properly read.
- Fixed transwrapping issues with some encoders.
- Fixed crash in Workflow Manager when deleting newly copied nodes.
- Fixed priority not being properly set on API job submit.
- Fixed ProRes encoder not working properly on all frame rates.
- General stability improvements and several other minor fixes.